{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大家好，这篇是有关台大机器学习课程作业二的详解，题目同Coursera。\n",
    "\n",
    "我的github地址：  \n",
    "https://github.com/Doraemonzzz\n",
    "\n",
    "个人主页：  \n",
    "http://doraemonzzz.com/\n",
    "\n",
    "作业地址:  \n",
    "https://www.csie.ntu.edu.tw/~htlin/course/ml15fall/\n",
    "\n",
    "参考资料:  \n",
    "https://blog.csdn.net/a1015553840/article/details/51085129  \n",
    "http://www.vynguyen.net/category/study/machine-learning/page/6/  \n",
    "http://book.caltech.edu/bookforum/index.php  \n",
    "http://beader.me/mlnotebook/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 1\n",
    "\n",
    "这一题和下一题就是learning from data第一章的Exercise 1.13\n",
    "\n",
    "错误有两种，一种是“去真”:$\\lambda\\mu$，另一种是“取伪”：$(1-\\lambda)(1-\\mu)$，所以犯错的概率一共$\\lambda\\mu+(1-\\lambda)(1-\\mu)$  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 2\n",
    "\n",
    "$$\n",
    "P=\\lambda\\mu+(1-\\lambda)(1-\\mu)=\\lambda\\mu+1-\\mu-\\lambda+\\lambda\\mu=1-\\lambda+(2\\lambda-1)\\mu\n",
    "$$\n",
    "所以当$\\lambda=\\frac 1 2$时，$P=1-\\lambda$，此时和$\\mu$独立"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 3\n",
    "\n",
    "这题就是learning from data第二章的Problem 2.12\n",
    "\n",
    "在此回顾下公式\n",
    "$$\n",
    "E_{out}(g)\\le E_{in}(g)+\\sqrt {\\frac 8 N ln(\\frac {4({(2N)}^{d_{vc}}+1)} \\delta )}的概率大于等于1-\\delta\n",
    "$$\n",
    "回到这题，题目中$\\delta =0.05,\\sqrt {\\frac 8 N ln(\\frac {4({(2N)}^{d_{vc}}+1)} \\delta )}=0.05$\n",
    "这个方程直接解的话不好解，可以作图看一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "452957\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "delta=0.05\n",
    "dvc=10\n",
    "\n",
    "def f(N):\n",
    "    return (8/N*math.log(4*((2*N)**dvc+1)/delta))**0.5-0.05\n",
    "\n",
    "n=1\n",
    "while(True):\n",
    "    if(f(n)<=0):\n",
    "        break\n",
    "    else:\n",
    "        n+=1\n",
    "\n",
    "print(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 4\n",
    "这题和下面一题就是learning from data第二章的Problem 2.20，注意这里多了Variant VC bound\n",
    "\n",
    "我们注意到(d),(e)左右两边都有$\\epsilon$，所以这里要处理一下。\n",
    "\n",
    "先看(d)，令$a=\\frac 1 N,b=ln{\\frac {6m_H(2N)}\\delta}$，那么不等式可以化为\n",
    "$$\n",
    "\\epsilon\\le \\sqrt{2a\\epsilon+ab}\n",
    "\\\\两边平方可得\n",
    "\\\\\\epsilon^2\\le2a\\epsilon+ab\n",
    "\\\\(\\epsilon-a)^2\\le ab+a^2\n",
    "\\\\ -\\sqrt{ab+a^2}\\le \\epsilon-a\\le\\sqrt{ab+a^2}\n",
    "\\\\a-\\sqrt{ab+a^2}\\le \\epsilon\\le a+\\sqrt{ab+a^2}\n",
    "\\\\这里只要考虑上界\n",
    "\\\\\\epsilon\\le a+\\sqrt{ab+a^2}\n",
    "\\\\将a=\\frac 1 N,b=ln{\\frac {6m_H(2N)}\\delta}带入可得\n",
    "\\\\\\epsilon\\le \\frac1N+\\sqrt{\\frac{1}{N^2}+\\frac{1}{N}ln{\\frac {6m_H(2N)}\\delta}}\n",
    "$$\n",
    "对于(e)，令$a=\\frac 1 {2N},b=ln{\\frac {4m_H(N^2)}\\delta}$，那么不等式可以化为\n",
    "$$\n",
    "\\epsilon\\le \\sqrt{a(4\\epsilon(1+\\epsilon)+b)}\n",
    "\\\\两边平方可得\n",
    "\\\\\\epsilon^2\\le4a\\epsilon(1+\\epsilon)+ab\n",
    "\\\\(1-4a)\\epsilon^2-4a\\epsilon\\le ab\n",
    "\\\\注意这里我们只要考虑N很大的情形，所以1-4a=1-\\frac 2N>0\n",
    "\\\\(1-4a)(\\epsilon-\\frac {2a}{1-4a})^2\\le ab+\\frac{4a^2}{1-4a}\n",
    "\\\\(\\epsilon-\\frac {2a}{1-4a})^2\\le\\frac{ab+\\frac{4a^2}{1-4a}}{1-4a}\n",
    "\\\\ -\\sqrt{\\frac{ab+\\frac{4a^2}{1-4a}}{1-4a}}\\le\\epsilon-\\frac {2a}{1-4a}\\le\\sqrt{\\frac{ab+\\frac{4a^2}{1-4a}}{1-4a}}\n",
    "\\\\这里只要考虑上界，因此\n",
    "\\\\ \\epsilon\\le \\sqrt{\\frac{ab+\\frac{4a^2}{1-4a}}{1-4a}}+\\frac {2a}{1-4a}\n",
    "=\\sqrt{\\frac{ab}{1-4a}+\\frac{4a^2}{(1-4a)^2}}+\\frac {2a}{1-4a}\n",
    "\\\\将a=\\frac 1 {2N},b=ln{\\frac {4m_H(N^2)}\\delta}带入可得\n",
    "\\\\ \\epsilon\\le \\sqrt{\\frac{ln{\\frac {4m_H(N^2)}\\delta}}{2(N-2)}+\\frac{1}{(N-2)^2}}+\\frac{1}{(N-2)^2}\n",
    "$$\n",
    "\n",
    "这里还有一个要注意的点，这里老师希望我们使用$m_\\mathcal{H}(N)$的上界为$N^{d_{vc}}$，因此\n",
    "$$\n",
    "\\\\ln(m_\\mathcal{H}(N))\\le d_{vc}ln(N)\n",
    "$$\n",
    "后面我们作图处理下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from scipy.special import comb\n",
    "from math import log\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif']=['SimHei'] #用来正常显示中文标签\n",
    "plt.rcParams['axes.unicode_minus']=False #用来正常显示负号\n",
    "\n",
    "dvc=50\n",
    "delta=0.05\n",
    "\n",
    "#计算lnm(N)\n",
    "def lnm(n):\n",
    "    s=dvc*log(n)\n",
    "    return s\n",
    "\n",
    "#Original VC-bound\n",
    "def f1(n):\n",
    "    result=(8/n)*(log(4/delta)+lnm(2*n))\n",
    "    result=result**0.5\n",
    "    return result\n",
    "\n",
    "#Variant VC bound\n",
    "def f2(n):\n",
    "    result=(16/n)*(log(2/(delta**0.5))+lnm(n))\n",
    "    result=result**0.5\n",
    "    return result\n",
    "\n",
    "#Rademacher Penalty Bound\n",
    "def f3(n):\n",
    "    k1=(2/n)*(log(2*n)+lnm(n))\n",
    "    k2=(2/n)*log(1/delta)\n",
    "    k3=1/n\n",
    "    result=k1**0.5+k2**0.5+k3\n",
    "    return result\n",
    "\n",
    "#Parrondo and Van den Broek\n",
    "def f4(n):\n",
    "    k1=1/n\n",
    "    k2=1/(n**2)+(1/n)*(log(6/delta)+lnm(2*n))\n",
    "    k2=k2**0.5\n",
    "    result=k1+k2\n",
    "    return result\n",
    "\n",
    "#Devroye\n",
    "def f5(n):\n",
    "    k1=1/((n-2)**2)\n",
    "    k2=(log(4/delta)+lnm(n**2))/(2*(n-2))+1/(n-2)\n",
    "    k2=k2**0.5\n",
    "    result=k1+k2\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD6CAYAAAC1W2xyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xl8VNX9//HXnX0m+8aShBA2EyAo\nCogsQsQquKCV4m6BVkWrUm3rgrUqWhdw+7Zqf+6K2haqtQpWgSoSiCCISNgJa5BskD0zk8x+fn/c\nyWQhgQAJk5mc5+Mxj5nM3Ln3hOV9zpxz72cUIQSSJElS+NAEuwGSJElSx5LBLkmSFGZksEuSJIUZ\nGeySJElhRga7JElSmJHBLkmSFGZksEuSJIUZGeySJElhRga7JElSmNEF46CJiYkiPT09GIeWJEkK\nWZs2bSoXQiSdaLugBHt6ejo//PBDMA4tSZIUshRFOdSe7eRUjCRJUpiRwS5JkhRmZLBLkiSFmaDM\nsUtSV+J2uyksLMThcAS7KZIEgMlkIjU1Fb1ef0rvl8EudXuFhYVERUWRnp6OoijBbo7UzQkhqKio\noLCwkH79+p3SPuRUjNTtORwOEhISZKhLXYKiKCQkJJzWJ0gZ7JIEMtSlLuV0/z2GVrDbK2DZXKiv\nDnZLJEmSuqzQCvaaw/D9G7DqmWC3RJI6jM1m45prrmH8+PHMnDkTj8dz3O3vu+++du23vdu1Jjs7\n+7ivjx49mr179wKwdOlSfvWrXwHw6KOPMnbsWK655hpsNlur7503bx45OTmn3LbjWbhwIQsXLuyU\nfYeS0Ar25OEw8lbY+BaUbA12aySpQ7zyyisMGjSIb7/9FqfTyUcffXTc7f/yl7+0a7/t3e5UTJky\nha+++gqAlStXMnnyZNatW0dubi5r167l0ksv5c033+y040vHF3pnxUx6BHZ8Cl/eD79aDprQ6puk\nru2Jz3ews7i2Q/c5JDmax6cObfP1DRs2cNtttwEwfvx4Nm7cyE033UR2djajRo1i69atrFixIrB9\ndnZ2YMRbX1/PtGnTqKysZMCAAWRlZfHHP/7xmO3mzZuH2+0mNzeX2tpali9fTmRkJNOnT8dutzNw\n4EDee++9dv9OkydP5oUXXuCuu+5i1apVPPbYY7z88stcfvnlKIrC5MmT2bFjR5vvf+mll3j88cfp\n3bs3//jHP9BqtcyZM4e8vDxiY2P54IMPWLJkCQCzZs0iJyeHnJwc0tPT2bJlC1u2bKG0tJSPPvqI\njIwMrrvuOqqrq9Hr9dx0003t/j3CVeilojkOLnkSDm+ALYuC3RpJOm1Wq5WIiAgALBYLtbVqx7J+\n/XrGjBnTLNRb2r17N6mpqXz77bfs27cvEOqt2bdvH2vWrGHatGl88803lJSUMGfOHL7++msKCgo4\ncuRIu9s8evRo8vLyKCwsxGKxkJCQwJEjR4iPjwegf//+TJ06lTvuuIPs7OzA7cknnwRg5MiRrF69\nmpiYGD7//HP++9//4nA4yM3N5Re/+AULFixo89gbN25kxYoVzJ07l6VLl/Kf//yHvn37smrVKvr2\n7dvu3yGchd6IHeCcG+HH9+GrxyDzcjXsJakDHG9k3Vmio6MD89F2u53o6GgAsrKymDZt2nHfm5KS\nwqZNm5gwYQL33nvvcbedMWMGAGlpabhcLvR6PW+//TbvvfcelZWV1NfXt7vNWq2WESNGsGDBAi69\n9NJjfo/vv/+e1atX88Ybbxzz3nnz5jF69GgAzjvvPPbv34/X6w08d8EFF/Dpp5+SmZkZeE/Ttt14\n443o9XrS0tI4cOAABw8e5JxzzgHUDkMKxRE7qNMvl78A9ZXwzdPBbo0knZbRo0cHpkxyc3M5//zz\nAYiMjDzhe5cvX86jjz7Kd999x80333zcbRs+FTR45513mD59OosWLTrmtfaYMmUKr7/+OlOmTAFg\n3LhxgXn31atXYzab23zvpk2bANi6dSvp6ekMHTqU9evXA+onlaFDh2IwGCgrKwNg2bJlbf4eaWlp\ngWmfzZs3n/TvEY5CM9gBep8No26HH96B4rxgt0aSTtk999zDgQMHGDt2LGazmWuvvbbd7z333HOZ\nM2cOkyZN4oYbbmD79u3tfu8ll1zCs88+y6RJkwAoKio6qXZPnjyZyMjIwEj7qquuon///owdO5bc\n3NzAmTKtyc3NZeLEiRw5coSrr76aK664ArPZzPjx4/nkk0944IEHmDRpEp9//jl33303Xq+3zX39\n4he/YM+ePWRnZ7Nnz56T+h3ClSKEOOMHHTlypOiQeuz11fDqSIhLh1//Ty6kSqdk165dDB48ONjN\nOCVvvfUWixYtQq/Xo9fruf/++094qqIUGlr7d6koyiYhxAnnm0Jzjr2BORYu+TN8difk/QPO+2Ww\nWyRJZ9Ttt9/O7bffHuxmSF1M6A9xz7kB0sbA149DXWWwWyNJkhR0oR/siuJfSK2Gb54KdmskSZKC\nLvSDHaBXFpw/G354F4rlqrgkSd1beAQ7wEUPQ0QSfPEH8PmC3RpJkqSgCZ9gN8XApU9B0SbY/GGw\nWyNJ7dZWQa2TcaoFv/Ly8sjLO/7pwrLgV+gJn2AHOPs6SBsLX8+TC6lSyGitoNbJOtWCX+0Jdlnw\nK/SE9umOLSkKXPECvH4hrHwSpnZedTspTC2bC6XbOnafvYbBZfPbfLm1glo2m63VAl3tKQzW2ntb\nKwL217/+lU8//RSADz/8kJUrV7a7fbLgV9cWXiN2gJ5DYfSdsGmhOi0jSV1cawW12irQ1Z7CYG29\nt2URsGeffZa5c+cyd+7cNkO9rfbJgl9dW3iN2Btkz4Xt/1YXUm9bCRptsFskhYrjjKw7S2sFtdoq\n0NWewmBtvbdlEbDTaZ8s+NW1hd+IHcAUDZc+rZ76+OMHwW6NJJ1Qy4JabRXoak9hsLbe21qhL7PZ\nTF1dHQDHKy8iC36FlvAMdoBh06HveFj5hFxIlbq8lgW1TqdA18m895JLLuE///kP48aNIzc3t93t\nkwW/urbQLgJ2Ikd3wWvj1BoyU//a+ceTQlIoFwGTwtfpFAEL3xE7QI/BcMFvYNP7UCgXUiVJ6h7C\nO9gBJj4EkT3hyz+Ar+2PeJIkSeEi/IPdFA2TGxZS3w92ayRJkjpd+Ac7QNYvIP1C+PoJsFcEuzWS\nJEmdqnsEu6LA5c+DywYr5wW7NZIkSZ2qewQ7NC6k/vgBHN4Y7NZIUsCsWbM499xzGTNmDNdeey1u\nt7td7ykoKOjwtnT01+plZ2czZswYxo0bx4wZM457KmN7FRQUMGvWrGbPffbZZ1RXV5/0vvr160d2\ndjZjx4497hWwpyKYX1HYfYId1IXUqN5yIVXqcl555RW+++47IiMj+frrr4PdnA718ccfs3btWgwG\nQ6f9bqca7FqtlpycHNauXcuHH36I3W7vhNadeeFZUqAtxih1IfXfv4ZN78Go24LdIqmLWfD9AnZX\n7u7QfWbGZ/LQ+Q+dcDshBDabDYPBQHFxMddddx2KojBhwgSefvppDh48yM0334zFYqG2thaAI0eO\nMGvWLGpqapg6dSoPP/ww/fv3Jzk5mZSUFPLz83nmmWfo06cPt956K3q9nptvvpm77rqL/fv3M3v2\nbKxWKxdddFFgxPr888/z6aefkpiYyJIlS9i7dy933HEHdXV13H333cyYMYOcnByWLFlCQUEBQ4YM\n4emnnz7h71ZdXY3ZbGbPnj3H7C87O5srrrii2XG3b99+TJtbc9lll7F582Z2797NuHHjePHFF7no\noov4+9//TkpKChdffDEffPABKSkpbbbP4XDgdrvRarUcPHiQ2267jfr6eq655hoeeOCBZkXWZs2a\nxbx585g1a9Yxbf7xxx+ZPXs2vXr1orS09IR/552le43YAYZOg34T1OqP9vJgt0aSAJgzZw7p6en0\n7NmTSZMmUVRUxPz581m2bBmff/45AM899xwPPvggy5cvx2q1AvDss89y/fXXs27dOj777DMqKioQ\nQvD+++9z5MgRXnzxRTZu3MiRI0d49913Wbp0aaBS5AMPPMBTTz3F999/j8/nC9R+iYiIYN26ddhs\nNkpKSnjwwQd5/PHHyc3NZcGCBYHSAx999BHPP//8CUP92muv5cILL2Tw4MFMmDChzf21PG5rbW7N\nsmXLmDJlCosXL+bFF18E4JZbbmHx4sWUlpZiMBjaDHWv10t2djb9+vVj6tSpmEwmHnjgAZ588knW\nrl3LihUr2LVrV5vHbtnmP/3pT7zzzjssXryYo0ePHvfPpTN1rxE7NH5H6mtj1S/AvvpvwW6R1IW0\nZ2TdGV555RW+/fZbjEYjiqKg0+l44okniIyMDIR4QwEsnU7H8OHDAcjPz+e7775j4cKF2O12iouL\n6du3L1qtlvT0dLRaLUIIhBA8+OCDJCYm4vF4ANi9ezejRo0CYMGCBWg06jhv5syZQGOxsPz8fB5/\n/HEURcHr9QamPG688UYGDhx4wt/t448/JjU1NfBzW/tredzW2txe1157LVdccQUGg4Fbbrmlze0a\npmLq6+uZOHEiubm57Nq1i9GjR6MoCiNHjmT37uaf4JoWKWvZ5oKCAs4++2w0Gg0ZGRkn1eaOdMIR\nu6IoMYqiLFMU5X+KonyqKIqhje3eURTlO0VR/tTxzexgSRkw5m7Y/Hc4/H2wWyNJANxxxx288847\neL1eXnrpJR5++GHefvttFEUBGgtgeb1etm1Ta8ZnZGQwf/58cnJymDt3bqCUbktPPvkkb731Fs89\n91xgATMzM5OG0h6TJ08OfEtSy8JbGRkZLFy4kJycHO655x4MBjUC2lOQrDVt7a/lcVtrc1uaFjMD\ntfpknz59eOutt7jmmmtO2Caz2YzFYsFqtTJkyBDWr1+PEIKNGzcyePDgQJEyu93erKZOyzb36dOH\nnTt3YrfbA3+ewdCeEfvNwEtCiK8URXkNmAIsbbqBoijTAK0QYoyiKO8qijJICBG836o9JjwIWz+G\nL34Ps1fL0r5S0MXFxTFp0iQ++eQTrrzySu68806SkpKwWCwUFRXx4IMPcsstt/DSSy8FwnDu3Lnc\neuut/OlPf6Jfv37ccMMNre572rRpXH755aSmpuJ2u3E4HDz33HPcfvvtuN1uLr74YgYNGtTqe+fP\nn8/tt9+OzWZj3LhxrVaJPBnt3V9rbTaZTK1uO2PGjMDv0lBtcvr06RiNRiwWS5ttaZiKsVqtZGRk\ncMkllzB48GBuvfVW6uvrmTZtGpmZmcyYMYOZM2eSlpZGVlZWm/t78sknmTlzJklJSW12smfCSRUB\nUxTl38ALQoj1LZ5/GVguhPhSUZQbALMQ4r0W28wGZgOkpaWNOHTo0Gk3/rTt+BQ+nqVOzZx/e7Bb\nIwWJLAIWfv7+97/zf//3f7z77ruB+u2h5owUAVMUZQwQ1zLU/SKAhtqglUDPlhsIId4UQowUQoxM\nSkpq72E715CfQ/9sWPlnsJUFuzWSJHWQW265hU2bNoVsqJ+udgW7oijxwCvAr9vYxAY0VNqPbO9+\ng05R4LLnwV2nLqRKkiSFgfYsnhqAj4GHhRBtzZ9sAsb7H58DFHRI686EpLNg7D2Q9w/4qbUPI5Ik\nSaGlPSPrW4HzgEcURclRFOVxRVGearHNZ8AvFUV5CbgO+KKD29m5JjwA0anwxf3gPbnTqiRJkrqa\nEwa7EOI1IUScECLbf3tCCPGnFtvUAtnAeuAiIURN5zS3kxgiYMozcGQb/PBOsFsjSZJ0WjpsLlwI\nUSWE+EgIEbzraE/H4KtgwCT45imwBe+KMan7mTdvXuCqzIsvvpji4uKgtKHhkvlgHGP06NGB876X\nLl163O9Q7cjjtldDobbs7Gyuv/760y5mtnDhQhYuXHha+zie0FjkPBMCC6n18NVjwW6N1M088sgj\nrFmzhl/96le88sorwW7OGTdlyhS++uorAFauXMnkyZOD3KJjvfLKK+Tk5BAXF8f//ve/YDfnuLpf\nSYHjSRwI434LuS/CeTOg79hgt0g6w0qfeQbnro4tAmYcnEmvP/6xXdtWVVVhNpux2WxMnz4du93O\nwIEDA7VSsrOzGTVqFFu3bmXFihWtPud0Opk1axbFxcWkpqby3nvv8cwzz+B2u8nNzaW2tpbly5dj\nNBq59tpr8Xq9CCHIzs5u9b0NF0M11Vr75s2b165jtGby5Mm88MIL3HXXXaxatYrHHnus3cfo1atX\nq3+OLY9bV1fHjBkzOHr0KMOGDeNvf/tbu/fXVHl5eeCiqvb82Xs8nmOO22DHjh3cc889LF26lKio\nqHb9G2kPOWJv6cI/QEwfuZAqnVFPP/00EyZMYP369dx7772UlJQwZ84cvv76awoKCjhy5AgA69ev\nZ8yYMYFQb+25t956i6ysLFavXs2gQYN49913Adi3bx9r1qxh2rRpfPPNN7z55ptceeWVrFq1Cr1e\nf9z3ttRW+9pzjNaMHj2avLw8CgsLsVgsJCQktPsYrWntuG+++SZZWVmsWbOGkpIStm7d2u79gVqo\nLTMzk+LiYsaMGdPuP/u2jltSUsLNN9/MokWLOjTUQY7Yj2WIgCnPwr9ugY1vwwV3BrtF0hnU3pF1\nR3vkkUeaFauqqqri7bff5r333qOysjJQeCorK4tp06Y1e2/L53bu3Bn4+YILLmDZsmXExcUxY8YM\noLFg1cGDB7n++usBGDlyZJvvbY1er2+1fe05Rmu0Wi0jRoxgwYIFXHrppSd1jNa0dtz8/HzWrVtH\nTk4O1dXVFBUVtXt/oE7FjB07lt/97nfMnz+fRx99tF1/9i6Xq9Xjvvrqq5x77rkcOnTohJ8STpYc\nsbcm80oY+DNY9TRYjwS7NVI39M477zB9+nQWLVrUrJZKa4W3Wj43dOhQ1q9Xr8lYv349Q4cOBY4t\nWNVQVAwgLy/vuO9tb/vac4y2TJkyhddff50pU6ac1DFa09pxMzIyuO+++8jJyeGpp54iLS2t3ftr\noNFoiIuLC1TcbM+ffVvHffTRR3nttdd49NFH2338drezw/cYDhQFLnsOPA65kCoFxSWXXMKzzz7L\npEmTAAKjvPa47bbb2LFjBxMmTGDv3r3HfI1cg9mzZ/PJJ5+QnZ0d+OKO9r63ve1r7RhtmTx5MpGR\nkYwePfqkjtHe495+++0sW7aMCRMm8Prrr9OnT5927w/UqZhx48bx1Vdfcffdd7e6TWt/fm0d12Qy\n0adPHzIzM1m6dGmr+ztVJ1UErKOMHDlSNJQL7dK+eQrWPA+zvoT0ccFujdRJZBEwqSs6I0XAuqXx\nv4eYNPjyfvCe+AuGJUmSugIZ7MdjsKgLqUd3wvdvBbs1kiRJ7SKD/UQyr4CBl8CqZ8AamhfVSpLU\nvchgPxFFgcsWgNcJ/+v41WtJkqSOJoO9PRIGwLj7YNtHUPBtsFsjSZJ0XDLY22v87yA2zX9FqlxI\nlTrWrFmzGD58OCNHjuStt+R6jnR6ZLC3l8ECUxZA2S7Y8EawWyOFoVdffZUVK1bwxBNPBC47l6RT\nEXIlBYQQKIoSnINnXAaDJkPOs5D1C4juHZx2SJ0m96M9lB+2deg+E/tEcuF1Z7Vr24SEBK644go+\n/fRTHnroIWpqapg6dSoPP/wwt99+O3fffTfDhw/njjvu4LbbbiMpKYlHHnkkUKir4fL7mTNnUl1d\nzYgRI/jLX/7CzJkz+c1vfsMFF1zArFmzuPPOO+nXrx+zZs1qdgwpPITUiH3fURtXvbqW/WUd+x+v\n3RQFLpuvTsX875HgtEEKewkJCfz5z3/m+uuvZ926dXz22WdUVFQwffr0QO2W3bt3M2rUKAA+//xz\n7rjjjkAFyGeeeYYbbriB3NxcampqWL58OTNmzOAf//gHLpeLXbt2ccEFF/Dss88ecwwpPITUiN1s\n0FJcXc/sD37gs7vHEWVqu1pcp4nvDxf+Xh21G6NhynzQm858O6RO0d6RdWeqrKzE6/Xy2muvsXDh\nQux2O8XFxVx88cW89tpr7Nq1q1lBrUsvvZQLLrgg8PPOnTu58061eN3o0aPZtWsX9957Lw899BBf\nfPEFV111FaAWxfruu++aHSMhIeHM/rJSpwipEXtKrJlXbzqPgoo6fv/RFny+M18OAYAL71cXUze9\nB+9Ngaq2vuNbkk5OdXU1y5YtY/LkycyfP5+cnBzmzp1LfHw8Op2OpKQkFi1axPTp0wPvaU8hKo1G\nwyWXXML9998fqCKZkZFxzDGk8BBSwQ4wZkACj1w+mK92HuHVVfuC0witDn42D274J1QcgDcnwt6v\ng9MWKWzMmTOHKVOmsGDBAhYuXMgLL7zAuHHjWL58OT179gTgmmuuYfHixc1G6C09/PDDLF68mPHj\nxxMbGxsog3vttdeSlpZG3759AZg7d26rx5BCX0gWARNC8IePtvBpXhFvzxjJxYOD+A+yYj98NAOO\n7ICJD8HEB0GjDV57pJPWHYqArVy5kgcffJCnn346UBZX6tq6XREwRVF4ZtowhiZHc9/iPA4EazEV\n1IuXbv0KzrkRVs+Hf1wLdrkIJXUtF198MZs2bZKh3k2EZLADmPRaXr9lBHqdhtkfbsLqCOJFQwYL\n/Pz/wdS/QkGuOjVTtCl47ZFOWjA+uUpSW07332PIBjtAapyFV286l4Pldv4QzMVUUE+FHDELfr0C\nUODdKbDxHZCB0eWZTCYqKipkuEtdghCCiooKTKZTP9suJOfYW3o79wBPfbGLP1xyFnMuHtRh+z1l\ndZXwn9mw7ys4+wa48v/UUb3UJbndbgoLC3E4HMFuiiQB6mAjNTX1mC8Ab+8ce0idx96WW8f3Y3tR\nDS99vYehKdFMygzy6r4lHm76CHJfUMv9lm6D6z9U5+OlLkev19OvX79gN0OSOkxIT8U0UBSFZ6ed\nzeBe0dy7OI+D5fZgNwk0GvUMmVs+AWsJvJkNuz4PdqskSeoGwiLYQb0q9Y1fjkCnUZj9wQ/YnJ5g\nN0k18GK4Yw0kDoJ/3aLWdPd2kbZJkhSWwibYAfrEW3j1pvPYX2bj/o+2dJ3FsNg+8KtlMOo2WPcy\nfHAVWI8Eu1WSJIWpsAp2gHEDE/nj5YNZvqOU/5ezP9jNaaQzwhUvwjVvQtGP8MaFcGhdsFslSVIY\nCrtgB3Ux9erhybzwv3xW7T4a7OY0d871cPtKMETCwith3avylEhJkjpUWAa7oijMn3Y2mb2i+e3i\nzRR0hcXUpnoOhdk5kHm5Wv73oxngqA12qyRJChNhGeygLqa++csRaDUKsz/sQoupDUzRcN2HcOlT\nsPsLeOsiOLIz2K2SJCkMhG2wg38x9cbz2HfUxgMfd6HF1AaKAmPnwMzPwWmFty+GrR8Fu1WSJIW4\nsA52gPGDEpl7WSbLtnexxdSm0sepp0Qmnwv/uR2++AN4nMFulSRJISrsgx3g9gv7M/UcdTE1J7+L\nLaY2iOoFM5bC2N/Cxrfhvcug+nCwWyVJUgjqFsGuKAoLfjGMjJ5R/HZRF1xMbaDVwaV/Vufey/bA\nGxNg38pgt0qSpBDTrmBXFKWnoii5x3k9RVGUQkVRcvy3pI5rYsewGHS8NWMkGo3CHR9uwt7VFlOb\nGnKVetZMVC/4+y9g9XPg8wW7VZIkhYgTBruiKHHA+0DEcTYbDTwthMj238o6qoEdqU+8hVduPJe9\nR6088O8uuJjaVOJAuO1rOPs6WPU0/PM6tWqkJEnSCbRnxO4FrgeOd6L1BcBtiqL8qCjKMx3Ssk5y\n4aAkHpqSyZfbSnl99YFgN+f4DBFwzRtwxUtwcDW8MRGKNwe7VZIkdXEnDHYhRK0QouYEmy0DsoFR\nwBhFUc5uuYGiKLMVRflBUZQfysqCO6CfPaE/V57dm+dW7Gb1ni754aKRosCoW+FXywEB71wKmxbK\nq1UlSWpTRy2erhNCWIUQXmAzcMy3XQgh3hRCjBRCjExKCu4UvKIoPDf9bDJ6RjHnnz9yqKKLLqY2\nlToCZq+G9PHw+b2w5G5w1QW7VZIkdUEdFewrFEXprSiKBbgU2N5B++00FoOON385EkVRF1PrXF14\nMbVBRALc/G+YOBfy/qmO3iu66Ln5kiQFzUkHu6IokxRFuafF008Aq4D1wOtCiPyOaFxnS0tQF1P3\nHLHywL+3du3F1AYaLVz0MNz8MdQWwmtj4Yv7oepQsFsmSVIXERbfeXq6XsvZz4Llu5l7WSZ3Tgyh\nr6+rPgyrF8CWxSB8MOxaGP876JEZ7JZJktQJ2vudp93iAqUTuXNif64Y1pvnlu9mTVdfTG0qtg9c\n/SrcuwVG3wG7lsL/Gw2Lb4bCTcFunSRJQSKDncbF1LN6RjFn0WZ+qgixRcmYFJjyLNy3HSY+BAW5\n8PYkeP8qOJAjz6CRpG5GBrtfhFHHG78cgRCC2R/+EBqLqS1FJMBFf4Tf7YBL/gxl+fDB1WrVyF3/\nlVevSlI3IYO9ib4JEbx847nkH7HyYKgsprbGGAXjfqtO0Vz5f1BXAf+6GV4bo87He93BbqEkSZ1I\nBnsL2Rk9eGByBv/dWsJbuV38ytQT0Ztg5K/hnk0w7W1QNPDpHfDyefD9W+CuD3YLJUnqBDLYW/Gb\niQO4fFgv5i/bTe7eEFpMbYtWB2dfC3euhRsXq8XFvrwf/jIMcl8Cx4kuLJYkKZTIYG+Foig8P/0c\nBvaIZM6izRyuDLHF1LZoNJBxGdz6P5j1BfQaBiufgP8bBiufBFsYdGKSJMlgb0uEUb0y1ecTzP5w\nE/Uub7Cb1HEURS1N8MtP1fLAA7LVkftfhsGXD8ov+JCkECeD/TjSEyP4643nsru0loc+CeHF1ONJ\nPheu+wDu/h6yfgE/vAMvD4dPf6OeVSNJUsiRwX4CF2X04P5LM1i6pZi3cw8GuzmdJ+ks+Pnf4Ld5\nMOo22PEp/G00/OsWKPox2K2TJOkkyGBvh7uyB3BZVi+eXbaLb/eWB7s5nSu2D1y2AH63HS78AxxY\nA29dBB/8HA6ukRc7SVIIkMHeDoqi8Py15zAgKZI5i34Mn8XU44lIhIsfVQP+Z0/AkR3w/lR4+2ew\n+wt5sZMkdWEy2Nsp0qjjzRkj8fgEd4TbYurxmKJh/H1w31a4/AWwH4XFN6lVJbf8C7wheIWuJIU5\nGewnoV9iBC/fcC67Smt58JOlgBqRAAAgAElEQVStuL3daNSqN8P5t8OcH+GaNwEBn86GV86DjW/L\ni50kqQuRZXtPwd9W7eP5FfkMSIrg0SuHkJ3RI9hNOvN8PtizDHJfhKJNYIxWz5EfcjUMuFi96lWS\npA7V3rK9MthPgRCCb3Yf5c//3UlBRR2TMnvwpysG0z8pMthNO/OEgIJvYetitdCYoxoMkXDWFBj6\ncxj4M3W0L0nSaZPBfga4PD4WrjvIyyv34fR4+dW4ftwzaSDRJn2wmxYcXrd65szOz9SQr68EfQSc\nNVkdyQ+6FAyWYLdSkkKWDPYzqMzq5PkVu/l4UyEJEQYemJzB9BF90GqUYDcteLwetS78ziWw63Oo\nKwe9BQZdAkN+roa8sRt+wpGk0yCDPQi2FlbzxOc72XSoiqyUaB6fOpRR6fHBblbweT3w0zrY8Zka\n8vajoDOp0zRDr1FH9MaoYLdSkro8GexBIoRg6ZZinv1yN6W1Dq46J5m5l2WSHCvnmQHweeGn79SR\n/M6lYCsFrdEf8j9XQ94UE+xWSlKXJIM9yOpcHl7P2c8baw6gKPCbiQO5Y2J/THptsJvWdfh8cHiD\nOie/cylYi0FrgAGT1OmajMvAHBvsVkpSlyGDvYs4XFnH/GW7+WJbCSmxZv54+WAuH9YLRenG8++t\n8fmgcKN/JL8EagtBo4cBF6kLrxmXg0VOa0ndmwz2Lmb9gQqe+Hwnu0pqOb9fPI9PHcLQZDnl0Cqf\nD4p/VAuR7VwKNT+BRgf9JqrTNZlXypCXuiUZ7F2Q1ydYvPEnXliRT029mxvOT+MPl5xFQqQx2E3r\nuoRQQ37nEnXxtfoQKFroN0EdyQ+eqta1kaRuQAZ7F1ZT5+avK/fywXcFmA1a7vvZWcwY0xe9VlZ4\nOC4hoGSLOie/4zOoOqh+j2v6eHVOfvBUiOyGVwFL3YYM9hCw76iVJz7fSe7ecgYkRfDY1KFMPCsp\n2M0KDUJA6Tb/nPxnULFPDfnew9WgTx8PaRfIM2yksCKDPUS0LE9wcWYP/nTlEPolRgS7aaFDCDi6\nU52PP7gaCn8An1sN+l5nqyHfdxz0HQPmuGC3VpJOmQz2EOP0eFm4toBXvlHLE/zaX54gqruWJzgd\nrjr1DJtDa9U6NoU/gNcJKNArC/r6R/R9x8pFWCmkyGAPUUetDl5YkR8oT/Dg5Eymj0hF053LE5wu\ntwOKfoCCtWqZg8KN4HGor/UYCunjGkf1ciFW6sJksIe4puUJhqXEMO+qIYzoK0eXHcLjVL/H9dC3\n6oj+8Pfg9n8rVlJmY8inj5eLsVKXIoM9DLQsT3D1cLU8Qe8YWZ6gQ3lcUJKnjuYL1qpXw7ps6msJ\ngxoXY/uOg+jewW2r1K3JYA8jdS4Pr/nLE2gVhd9kD2D2BFmeoNN4PepplQW56jz9T+vBWau+Fj9A\nnbrpO169j0kNblulbkUGexg6XFnHs8t28eW2UlJizTxyxWAuy5LlCTqd1wOlW/2LsWvVSpWOGvW1\nuPTGxdj0cRCbFtSmSuFNBnsY+25/BU98voPdpVbOTYvl2hF9uHxYL2IthmA3rXvweeHIdjXkG868\ncVSrr8WkqQGfMgJ6nwM9s+SXi0gdRgZ7mPN4fSzeeJh31x7kQJkdvVYhO6MHVw9P5meDe8ppmjPJ\n51PPo28I+UPr1C8WAfVc+sSz1JDvfY56Xn3vs+WFU9IpkcHeTQgh2FFcy2ebi1i6pZijVieRRh2X\nDu3Jz4enMHZAAjpZquDMEgJqi9R5+pKt/vstalniBnH9GsO+4SZPtZROQAZ7N+T1CTYcqOCzvCKW\nbSvF6vSQGGnkyrN78/NzUzgnNUbOxweT7ag/6PPUOfuSLVBV0Ph6dErzoO91NkQng/w7k/xksHdz\nDreXnPyjfLa5mG92H8Xl9ZGeYOGq4Sn8fHgy/ZPk9412CfVVas2bpqP78j2A//+lJfHYkX1cugz7\nbkoGuxRQU+9mxfZSPssr4rsDFQgBZ6fGcNU5yVx1TjI9ok3BbqLUlNMGR3aoIV/qn8Y5ugt8HvV1\nY4w6T9807BMGgkauq4S7Dg12RVF6Av8WQlzYxut64D9APPCOEOLd4+1PBnvwHKl18PmWYpbkFbOt\nqAaNAmMHJHLV8GSmZPUiWtam6Zo8TnWBtmG+vmSremZOQ2kEvUU9A6dp2Cdlgk6eKRVOOizYFUWJ\nAxYBPYQQ57Wxze+BaCHEPEVRvgSuF0JY29qnDPauYd9RG0vziliypZhDFXUYdBouzuzB1cNTuCgz\nCaNOjgC7NK9HnbZpCPvSrWrgu/z/9TR6iO8PSWdBYoZ6dk7SWerVtEY5FReKOjLYowEFWCKEyG5j\nm6XAXCHETkVR5gIbhBCr2tqnDPauRQhB3uFqluQV89+txZTbXESZdFye1Zurz01mdL8EtLIIWWjw\n+dQvICnJU+fuy/ao4V95AIS3cbvoVEgcBEkZ6n1D8Ef2kPP3XViHz7EripJznGBfCUwTQtQoijIb\nqBVCLG6xzWxgNkBaWtqIQ4cOteu40pnl8fpYu7+CJZuLWLGjFLvLS69oE1PP6c3Vw1MYmhwtz6wJ\nRR6XGvhl+VCeD+V7/Y/3gtveuJ0pRg34xAz/SN9/i+0LWl3w2i8BZz7YlwB3CCFK/dMypUKIf7a1\nLzliDw31Li9f7zrCkrwicvLL8PgEA5Ii+PnwFK4enkJagryiMuQ1nHNfvsc/um8S+vajjdtpDeoC\nbdPRfdJZ6nMG+aUwZ8qZDvbHgJ1CiH8rivI+8IYQYl1b+5LBHnqq7C6+3F7Cks3FfF9QCcC5abH8\nfHgKV5zdm0T5hdzhp75KDfnyPY2j+/J89dx74WvcLiatybSOf4SflAGWBDmt08E6LdgVRZkEDBFC\nvNrktb7Al8DXwFjgAiGaTug1J4M9tBVV17M0r5gleUXsLrWi1SiMSo9jdL8ERveP57y0OFnSIJy5\nHeqcfbMpnT3qY09943bmOP/ofqB67n1cP/99ugz9U3TGz2NXFCUZGA+sEELUHG9bGezhI7/UypK8\nIlbvKWNnSS1CgEGr4Zw+MYGgH9E3DotBzs+GPZ8Pag43juwbpncq9oK9rPm2hsjGkG95i+kDenlt\nRWvkBUrSGVdT7+aHgko2HKxkw4EKthfX4vUJdBqFYan+oO8Xz8j0OPldrt2Nyw5Vh9RpnGr/fdNb\nw/n4DaKS2w7+bnzmTlgGe6Wjknu/uZf7RtzHiJ4jOqFlUkeyOT38UFDJ9wfVsN9aWI3bK9AoMDQ5\nhtH94hndP4Hz0+OJscig77aEANuRJkHfIvibFk8D0JmbBH3f5qEf2zesyySHZbDvrtzNfavuo8hW\nxI2ZN3Lfefdh0YfvX2K4qXd5+fGnKjYcqGD9wUryfqrG5fWhKJDZK5rR/eK5oH885/dLID5CXjEp\n+bkdUP1TY9C3HPE3fI1hg8iex47yY/uqX4IS1TukT9sMy2AHqHPX8fLml/nnrn+SHJnM42MeZ0zy\nmA5uoXQmONxe8g5Xs+FAJRsOVvDjT1U43OrZFmf1jAzM0Y/ul0BSlDzrRmqFEFBX0SToDzYZ8R+C\n2sLmZ/AoGojsBTEpajXNmFT/vf/n6BS1Y9B0zVLXYRvsDTYf3cxjax+joLaAaYOm8YeRfyDaEN1B\nLZSCweXxsbWwmg0HK1l/oIJNh6qoc6knV/VPimB0vwQu8Ad9rxi5uCa1g8elLuhWHYTqw1BbrJ63\nX1Povy9qfiYPgEanzvEHwj9FvVI3OrnxcURiUOb5wz7YARweB69teY2FOxaSaErksTGPMbHPxA5o\nodQVuL0+thfVsOGgOk+/8WAlVqda4bBvgkWdo/eP6lPj5JScdAqEUM/XDwR9YZPwL1JH/LXF4HU1\nf5/W6A/6piP+ZDX0GzoEc1yHh3+3CPYGO8p38Oi6R9lbtZcr+l/BQ6MeIs4U12H7l7oGr0+wq6SW\n9QcqAmFfU+8GICXWzOh+8QxPi2Vw72gye0XJM2+kjuHzqV912HSUX1vov/d3ArXFzWvxgFpxs+mI\nvyH8e2ZB6gmzuVXdKtgB3F43b297mze3vkm0MZo/jv4jl/a9VNY1CWM+nyD/iJUNTYK+wt44suoT\nb2ZI72gG+29DekeTGmeW/yakjufzqmf2NAv9JlM+tcVgLQUEZE2H6e+c0mG6XbA3yK/M57F1j7Gz\nYic/S/sZj1zwCIlm+V2S3YEQgpIaB7tKav03K7tKajlYYafhn3mUUUdm76hmYZ/RK0peKSt1Pq8b\nrCXq9E9c31PaRbcNdgCPz8MHOz/gb5v/hkln4qHzH2Jq/6lypNZN1bk85Jda2VViZWdJDbtKrOwu\nqcXuX5jVKNAvMaJZ2A/uHU3PaKP8NyN1Kd062BscrDnIY2sfI68sj/Ep43l8zOP0iujV6ceVuj6f\nT3C4qo5dJbXs9I/sd5XUUljVeIZEnEXPkORoBvdqnM4Z2CMSg65rngonhT8Z7H5en5fF+Yv5649/\nRaNo+P2I3zP9rOloFPmfUzpWTb2b3U2nckpryS+14vSo50LrtQoDkiIDo/ohyeq9vKBKOhNksLdw\n2HqYJ9Y9wYbSDZzf63zmjZ1Hn6g+Z7QNUmjyeH0UVNgDI/udxWrwH7U6A9v0jDYGRvWDe0czICmC\n9IQIIoyhe5Wj1PXIYG+FEIJP9n7CCz+8gE/4+O25v+XGzBvRym93l05Bhc0ZWKBVp3Rq2XfUhsfX\n+H+qR5SR9MQI+iVEqPeJFtITI+gbH4HZIP/dSSdHBvtxlNpLefK7J8ktyuWcpHN4ctyT9I/pH7T2\nSOHD5fGxv8zGgTI7BRV2DpbbKShXH5fbml/k0jvGRHrTwE+IoF9iBH3iLfIsHalVMthPQAjBfw/8\nl/nfz8fhcfCb4b9h1tBZ6DTyo7PUOawONwXldRys8Id9uT3wuKrOHdhOUSA5xky/xAjSmwR+emIE\nfeIscvG2G5PB3k7l9eU8s+EZvjr0FUMShvDk2CfJiM8IdrOkbqamzh0I+YP+EX7D41qHJ7CdVqOQ\nEmv2T++o0zoNUz2pcWZ0Whn64UwG+0n6X8H/eHrD09Q6a7nt7NuYPWw2eq28JF0KLiEEVXXuZlM6\njcFfh83ZGPo6jUKfeAvp/sDvlxhBSqyZZP8t2qST5+WHOBnsp6DaUc2CjQv474H/MjB2IH8e92ey\nErOC3SxJapUQgnKb65i5/IPldRSU26l3N69dEmnU+YPeFAj7xuA30TPahF6O+Ls0GeynYU3hGp74\n7gnK68uZOWQmdw2/C5NOlomVQocQgjKrk6LqeoqrHRRX11PkvxX7b03n9UG9ArdntKnZKD+lSScg\nR/3BJ4P9NFldVl784UU+2fsJfaP78uTYJzmv53nBbpYkdZg6lycQ+g23wsBjByU19bi9zfNBjvqD\nSwZ7B1lfsp556+ZRbCvmxswbufe8e+XX8Undgs8nKLed/qg/OdZEjygjSVHqfY9oI0adPJ3zVMhg\n70Dy6/gkqXWnMuoHiLXo1ZD3h31SdOPjntGNHYDFIE8/bkoGeyf48ciPPL7ucQpqC7hqwFVMGzSN\nc5LOkee+S1IbfD5Bud3J0VonZVYnR60OjtY6Oep/fMT/fJnVicvrO+b9kUadf7RvpEe0iZ7+wO/R\nZPSfFGXqNnP/Mtg7icPj4P9t+X98uPNDPD4PscZYLky5kOw+2YxLGUeEPiLYTZSkkCOEoLrOHQj8\nhvA/Uuto7BCsagfR8mwfAJNe0yzse0SZ1M6goUOINpIUaSTWYkCrCd0OQAZ7J7O5bKwtXkvO4Rxy\ni3Kpcdag1+gZ1WsU2X2yyU7Npndk72A3U5LCihACm9PTPPRrmwd/w2Nrkwu7GigKxFkMJEQYiI8w\nkBhpJD7CQEJkw3PGwOOESCOxZj2aLtQRyGA/gzw+D3lH88g5nENOYQ6Hag8BkBGXoYZ8n2yGJAyR\npYIl6Qyqd3mbBX6Z1UGl3UWF3aXe21xU2J1U2F1Ut1gEbqBp6AgiDf4OwBjoFBoeJwQ6BiMxndwR\nyGAPooM1B1l9eDWrDq8irywPn/CRZE5iYp+JZKdmM7r3aHlevCR1IR6vj8o6NfArbS7K7S4qbWro\nV9hdVNicTToDV+BL1FvSahTiLHoSItr+JJDu/7auUxGWwS58PqzLlxP5s5+hMYTGFxtUOar4tuhb\nVh1exdqitdR56jBpTYxJHkN2n2wmpE6Q38kqSSHG7fVRFQh9/8jf5gp8Igh0BP7HTev9TD0nmVdu\nPPeUjhuWwW7/7jt++tWv0SYmEnfTjcTdcAO6+PhOaGHncHld/FD6A6sOr2J14WpK7CUoKAxLGsZF\nfS5iYupEBsYO7Bar+5LUnbg8Pqrq1E7AqNcwICnylPYTlsEuhKDuu++oWLgQ+5pcFKORmKuvJn7m\nDIwDBnRCSzuPEII9VXvUkD+8mu0V2wFIiUxRQ77PREb0HIFeIwuRSZKkCstg97i9fL/0IIPH9cZi\nL6Xy/Q+oWbIE4XQSMeFCEmbNwjJmTEiOeI/WHWV14WpWH17N+pL1OL1OovRRjE8Zz8Q+ExmfMp4Y\nY0ywmylJUhCFZbAX761iyV/y8HkFqZlxDJuYSmqqhpqP/0XVP/6Jt6IC41lnET9zJtFTrwyZefiW\n6tx1bCjZQE5hDjmHc6h0VKJVtJzX8zyyU7O5qM9F9ImW39cqSd1NWAY7QF2ti51ri9mxpghblZPI\nOCNDL0whc1QCnjVfUfn++zj37AnZefiWfMLH9vLtgVMp91btBaB/TH/1oqjkcWQmZBJtOLVVdkmS\nQkfYBnsDn9dHwbYKtuUUUri7Co1WYcB5PciamEL0kR1Uvv9+4zz8VVep8/ADB3bQbxA8hdZCVheu\nJudwDj+U/oBHqKvtyRHJnBV/FpnxmWTEZZARn0FKZIo8d16SwkjYB3tTVaV2tq8pYvd3pbjqPST2\niSRrQgp9E+uwLvp7s3n4+JkziRg7NiTn4VuyuqxsKdtCfmU++VX55FfmU1BbgE+oNTci9BFkxGVw\nVpw/8OMzGBg7UJ5DL0khqlsFewO308ue70vZllNERZENg1nH4DG9GTw8Et/Xn1L1z0V4y8vDYh6+\nLfWeevZX7ye/Mp/dlbvZU7WH/Kp87G47ABpFQ3p0OhnxGYGRfWZ8pjyXXpJCQLcM9gZCCEr217A9\np5D9m8vweQV9hsQzdFxP4grWU/3+wsZ5+BtvIO7GG0N6Hv5EfMJHka0oMLLfXbmbPZV7KLYXB7aJ\nN8U3m8bJiMsgPSZdVq6UpC6kQ4NdUZR3gCHAF0KIp1p5XQcc8N8A5gghtrW1vzNZUsBe42TX2mK2\nrynGXu0kMl5dbO1nLML+rw+wr16DYjA0ng8fBvPw7VXjrFFH9E2mcvZV78PtUy+XNmgMDIwbSGZ8\nJmfFnRUI/ShDVJBbLkndU4cFu6Io04CrhBCzFEV5F3hWCLG3xTbnAdcLIR5qT+OCUSvG5/VxcGs5\n23KKKMqvQqNTGDiiB5kDFbT/+xe1S/3z8BdeSPys8JmHP1lun5uDNQfVsG8S+FXOqsA2KZEpjSN7\n/+g+JTKlW/55SdKZ1JHB/jKwXAjxpaIoNwBmIcR7Lba5C7gbsAPbgDuEEMfWzPQLdhGwyhI721cX\nsXt9CW6Hl6S0KIaMiiNxz9fULvqHOg8/aBDxs2YRfeUVaIzGoLW1KxBCUFZfFpiz3125m/zKfA7V\nHkKg/vsxaU0kRyaTEplCSmQKqVGpgccpUSnydExJ6gAdGezvAC8LIbYoinIpcJ4QYn6LbUYBhUKI\nEkVRPgD+LYRY2mKb2cBsgLS0tBGHDh06ud+oE7gcHvZsKGXb6iIqi+0YLToyR/egrzsf98fv4czP\nR5uQoJ4PH+bz8Keizl3Hvup97K7czaHaQxTZitSbtQir29ps2yhDFKmRzcM+JTKF1MhUkiOT5Zk6\nktQOHRnsfwUWCSHW+6dlMoUQz7TYxiiEcPof/xbQCyFebGufwR6xtySEoGRfNdtyijiwuQyfT5A2\nJJ5BvWyYV/6dutWr/fPwVxE/c2a3moc/VTXOmmZBX2grDPxcbCvG6XU22z7RnNgY+i1G/L0ieslF\nXEmiY4N9BtBDCPGCoihPAPlCiH+22OYj4GlgO/AV8IwQ4uu29tnVgr0pe7WTHd8WsyO3iLoaF1EJ\nJjKzzPTY+QWOpf8OzMPH3XwTlpEj0UaeWpW27swnfFTUV1Bk8we+taixE7AVUWovxSsav/5Mq2jp\nFdGrWfCnRKUEPgEkmhPl/L7ULXRksEcDucBK4DLgBuBaIcSfmmyTBfwTUIClQohHjrfPrhzsDbxe\nHwfzytmWU0jx3mq0Og0DhsWSZtuM8tm7eMvKQVEw9OuHKWso5qxhmLKyMA3ORGM2B7v5Ic3tc3PE\nfiQQ9IXWwmbBX15f3mx7o9YYmN/vaelJkiWJRFMiiZZEksxJJJoTSTQnYtCG1zULUvfT0ac7xgGX\nAGuEEKWn27hQCPamKoptbF9dRP76UtxOL0lpkQzsXU901V4Mezfj3L4NT1mZurFWi3HgQEzDshrD\n/qxBKGF2IVQw1XvqKbGVNE7vNBnxH6k7QpWjKrCo21SMMabVwA889j8fqY+UnwCkLqlbX6DUWVz1\nHvI3lLItp5Cq0joAdAYNiamRxCdoiRGVRJTvw7DvR9zbtuCtqQFA0esxZmZiHpaFKWsYpqyhGAcM\nQNFqg/nrhC2Pz0Olo5Ky+jLK68opry9XH9c3eex/3uVzHfN+k9ZEgjmBJHMSSZYkEkwJJFmSSDIn\nNXs+zhiHViP/DqUzRwZ7JxJCUFlip/wnK2U/2Sg7bKXssBW3Q50X1mgV4pMjiI/XEOurIKJiP8Z9\nm/Ds2ILPrl7ar5jNmIYMwZyVhSkrC/OwLPRpaSgaWbTrTBFCUOuqbTXwW3YEVpf1mPdrFA3xpvhm\ngd/wKSDeFE+MMYY4UxyxxlhijDGYdXKKTjo9MtjPMOET1JTXU/aTlfLD/rD/yYrDpl7FqSgQ29NC\nfLxCjK+CiIoDmPdtxLtzC8KpniGiiYpqNl9vHpaFrndvOS3QBTg8DiocFZTVtRj5Nzz2P1/hqAgU\nYWvJpDUFwj7GGEOsMTZwa/pcnDEusJ1FZ5F//1KADPYuQAiBvdpJ2U9qyJcdtlF+2IqtqvFUv+gE\nE/FxqNM4Ffsx792IyN8CHvX6Lm18vDpfPzRLvR82DF2iLNjVVXl9XqqcVVQ7qql2tri1eK7GWUOV\ns4paZ22rawIAOo0uEPRNO4DjPRdliJLlmsOUDPYurN7qCozoy35Sw76mrD7wuiVaT0IcRPsqiaw8\ngHn/RpT8PBT/35WuVy91vn6oehaOPjUVfXKyPBsnRHl9XmpdtY1h76g6tgPwP9fQGdQ4a5qdEtqU\nRtEEpn+iDFFEGaKI1kcTZYgi0hCp/myIDrwWqY9s9rNRa5SfErooGewhxlnvofywfxrnJ3XOvqrE\nTsNfj9GiIz5WECOqiKzcj2n/D+j3bUFpMtLTJiSgT0nBkJqCPqXFLTkZjUle3RkuhBBY3VZqHDVU\nO6sDYd+yA7C5bFhdVqxuK1aXlVpXLR5fm9U+APVTQiDo9VGBDqFp+LfWITTc5PRR55HBHgbcLi8V\nRTZ1kdYf+BXFNnwe9e9Mb9AQG6tg0ToxeWox1ZdjqCpGd/QAusN70TlqafrfS5uYiD4lGUNKCvqU\n1BbB31sGfzcghMDpdTYL++Pdat21jZ2D/+bwOo57DK2iVT8Z6JuEvd5ChD6CCF0EEfoILHoLkfrI\nwOMIvf95nYVIQ2RgO71Wf4b+ZEKDDPYw5fX4qCq1B6ZxqkrtWCsd2CqdeD3NF+30Bg0REWDRuTD7\nbBjrKzDUlGA4WoCueB+Gugo0TRb6tEmJGJL9QZ/aNPiT1RF/Ny+GJqncXnezTqHW1Tz8a121WF1W\nbO7G5+xuO3a3nTpPHXa3/ZiSEm3Ra/SNoe/vDCx6SyD4W+0kmrzW9GbWmUP+k4QM9m5G+AT1NjfW\nCgfWSoc/7Js+duKwu5u9R1HAEqFg0bsxCzsmZxXG2lL05T+hK9mPyV6Grsl/QF1SUmPYp6aqgZ+S\ngiElBV2vXnLEL7Wb2+emzl0XCHy7206duw6b29asA2h5C7zHY8fuUu/r3HVtLj63ZNaZm90sOgtm\nfYufGx7rLcc81/K1htuZWpeQwS4dw+XwYKtyNgv8htC3VjiwVTsRvub/HgxGiDB4MSt1mJzVGG1H\nMFQcRld6AFNdOQaXNTDPr7FY0CYkoEtIUO/j49EmJqCLT0CXEI82IdF/n4A2Jkaesy91CJ/w4fA4\nsLvt2Ny25h2Gp7HDqHPXUe+pp95TT52njnp3feDnwHNNfm7rtNXWaBRN651Gw8/6xp+HJgzl8v6X\nn9Lv2t5glyXzuhGDSUd8bx3xvSNafd3nE9TVqCFvrXKoYV/pDHQAZa5Y3Po+0Gsk9FLfo9GAxejD\nrHFiEPXoXTb0jmp0lZXoCsrQVm3D4KxF77ahd9sap360WrTxcf7Qb9IRBDqGeHRNOgk5DSS1RaNo\nsOgtWPQWkkjqkH02rEU0Dfp6T32zzqFpZ3BMp+F/bPfYKasva/aeirSKUw729pLBLgVoNAqRcSYi\n40z0bmMbZ50bqz/sm47862pc1Fld1NvcOLxuiEG99Wn+foNeYNR6MCgujN469C4r+vpqdAcq0Obt\nRW/bgN5lxRDoCPxX80ZEBEb/auj7R/+BTwP+TiA2Fm1UlKzNI50WRVEw6UyYdCbiiOvQfZ+JWRIZ\n7NJJMVr0GC16ElPbLlfs8wkcNjf1NhcOq5t6m5t6q0u92dzUW904bOrjKqsLh+JGmKG1wZZe58Oo\n8WDEid5rR++0oq+qQvdTBbqaA806AYPLhsb/xV2K2Yw2OhptdBSa6Ji2H8dEo42ORhMVHXismEN/\nkU3qus7Evy0Z7FKH02gULNEGLNHtGzULn8BR1yTwm3UGagfR8Fqt/zURJSD52H1pNT4MGi96xYNO\nONF7HWjddejsNrTlVk0z6toAAAedSURBVLT1lWjtB9B56tF56tB76v2P1ZtGeECvRxsVpQZ+dLS/\nI4hGEx2F1t8ZaKKaPo5u0kFEybUDKehksEtBp2gUzJEGzJEGoPX5/6aET+Cs9wQ+ATiahL+zzo2z\n3oOrzoOz3oOzzoOj3hN4zuc7/sdgreJDr/Gix43O50DndaBz16EttaIrsKKxH0LvtjfrDHSeumYd\ngzYyEk1kJJoIizqFFBGBJiICTUSk/77lzb9d5LGvywqg0qmQwS6FHEWjYIrQY4rQn9TspxACj8uH\ns86Ds94dCH+XvwNo2iG46hs7hvp6T6DDaLg4rC0axYde8aLDjdbnRudzovU60NY60JTXoXXWoXGU\novU40HmdaL1OtB4HWq8TnVe9b7w50Bp0ascQqQa91tJaxxDRrCNp2pkoZjMaiwWNyYTGbJZrD92E\nDHap21AUBb1Ri96oJTLu5M+yEULgdfuO3xnUqZ2A2+nF5fDidqo3p8MTeOxyeI85rbTNNiPQKR50\neNAKt9oBuJ1oy+rRFNehddWhdRQ3dgT+TqGxk3Ch8bn89260ihedXovObFRD339TzCY0ZssxjzVm\nk387yzGPNWYzismMxtKwDzOKXi/XJ7oAGeyS1E6KoqAzaNEZtETEnPrpl0IIvB6fGvQOb5NOwNP2\nz4FtGzsIR0PH4fDgcbX/nGsADT60eNH6OwyNz43W4UJjd6H1ONC4HWjc9Wg81c06h2aPfS40Xrf/\nXv1Zi0f9MzJq0ZqMaM1mNCYTismEYjKiMZpQjEY0JiOK0RS4D7xmMqrbG00oRoP/ceNzGpMRxWRC\nY1Tv5VRV62SwS9IZpiiKOmrWazF30Heh+3wCT+ATgaex03B58bh8eNz+e5cXj8uL29nkccPzbp+6\nD5cXZ5Nt1fefXMcBoDR0HsKDRnjQeD1obG60NS4Ur9oZaNxONF4bGl8VWq8bjc+Dxudqdq/1qR2P\nxuu/97kbn9MIdFoFrUGLTq9BZ9ShMRkaOwmjqbEjaLgZ9OrPBiOKwYBiNKAYDP7nDOrzRgMag8G/\nvSGwrcbY9DlDl10ol8EuSWFAo1EwmHX/v727CY2jDuM4/v3tSyDUUiNKRJHaghdRrFLUYpUiWvBQ\nBRFb1HqoF6HoUSpVBC+KiB6EgJUqgl70IFioWC/SKqXQ4k09eGgRoeALtfUt2Zl5PMy/7q4mZpPd\nNsnM7wNhZ56dsDu/ZJ9nZyYvjI23WMXof5kriigb/0zPsPj3YEi3nene+ws6nZy8U5DNFORZWS9v\ny/XpmZy80x0g+TzXMeYjijRM8vKI4lxG40yHRjGD8jQgsmlUnCsHTpGlgVEua5Za73Y6XysyGk1o\nNUWj3aDZEs12g1a7SbPdojHWSgMlDYk0HMY3bGBi+8Mj+srMzo3dzOalRvf6xIX+q/8RQZFFeZTR\nKcqh0HObzTMospm0bVaQz+RkPfflne5t1sl7lguKvBwqxcIPTubaERrk5YD5K6PxZzkMrj72B1u3\nj+gx5uDGbmbLiiSa7fLd71L8IYkoymsg5Uf0D4Se5axTUGTdbbOebYqse/TxTy0Nm8l1ay74Prix\nm5n1UKN7kXylWp5n/s3MbNHc2M3MKsaN3cysYtzYzcwqxo3dzKxi3NjNzCrGjd3MrGLc2M3MKkYX\n4//v/edBpR+BUxf9gUfrcuCnpX4Sy4jz6Oc8upxFv2HyWBsR8/7H7iVp7FUg6XhEbFzq57FcOI9+\nzqPLWfS7GHn4VIyZWcW4sZuZVYwb++LtW+onsMw4j37Oo8tZ9Lvgefgcu5lZxfgdu5lZxbixm5lV\njBv7LCStkfSJpEOSPpI0Jmm/pKOSnuvZbqBaFUialPRVWq51FgCSpiRtS8u1zUPShKSDko5LejPV\napdHen0cScttSQckfSlp17C1xXBjn92jwGsRsRU4DewAmhGxCVgv6TpJDw5SW7I9GL1XgfFB97vK\nWUi6E7gyIg44D3YC76efy14t6RlqloekCeBdYFUqPQWciIg7gIckrR6ytmBu7LOIiKmI+CytXgE8\nBnyQ1g8Bm4EtA9ZWPEl3A79TDrkt1DuLNvAWcFLSA9Q8D+Bn4AZJlwLXAOuoXx45sB04m9a30N23\nw8DGIWsL5sb+PyRtAiaA74EfUvkXYJJyOg9SW9EkjQHPA3tSadD9rlwWyePA18ArwK3AbuqdxxfA\nWuBp4BtgjJrlERFnI+LXntIwr5GR5OLGPgdJlwFvALuA34DxdNcllLkNWlvp9gBTEXEmrdc5C4Cb\ngX0RcRp4j/JdVZ3zeAF4MiJeBL4FHqHeecBwr5GR5FKlMEcmvUv9EHg2Ik4BJ+geKt4EnFxAbaW7\nB9gt6XNgA7CN+mYB8B2wPi1vBK6l3nlMADdKagK3AS9T7zxguH4xklxai/mkGngCuAXYK2kv8A6w\nU9JVwH3A7UAARwaorWgRcdf55dTc72ew/a5cFsl+4G1JO4A25TnRj2ucx0uUr4+1wFHgder9/QHl\nhdSD6SL79cAxytMri60tmH/zdEDpyve9wOF0GD5wrWqcRT/n0c95QBpYm4FPz59/H6a24Md3Yzcz\nqxafYzczqxg3djOzinFjNzOrGDd2M7OKcWM3M6uYvwEwZWhX0WtujgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2148438acc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#产生点集\n",
    "import numpy as np\n",
    "x=np.arange(1000,10001,1000)\n",
    "\n",
    "y1=[f1(i) for i in x]\n",
    "y2=[f2(i) for i in x]\n",
    "y3=[f3(i) for i in x]\n",
    "y4=[f4(i) for i in x]\n",
    "y5=[f5(i) for i in x]\n",
    "\n",
    "#作图\n",
    "plt.plot(x,y1,label=\"Original VC-bound\")\n",
    "plt.plot(x,y2,label=\"Variant VC-bound\")\n",
    "plt.plot(x,y3,label=\"Rademacher Penalty Bound\")\n",
    "plt.plot(x,y4,label=\"Parrondo and Van den Broek\")\n",
    "plt.plot(x,y5,label=\"Devroye\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到Parrondo and Van den Broek以及Devroye非常接近，我们再仔细看一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD6CAYAAABApefCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd4VVW6+PHvykkjPaQSSAi9Siih\nJBSxIOiAOgjCKM1RgRHRmXvnN+pF7zTrXK4zd3REUQSxoKgjgghig4DUhF6lBUgB0kNCIG39/tgn\n/YTsQJKTnLyf5zkPJ/usdfbKfsK711577XcprTVCCCEcm5O9GyCEEKLxSbAXQohWQIK9EEK0AhLs\nhRCiFZBgL4QQrYAEeyGEaAUk2AshRCsgwV4IIVoBCfZCCNEKONu7AWUCAwN1ZGSkvZshhBAtSkJC\nQrrWOqiucs0m2EdGRhIfH2/vZgghRIuilDpjppwM4wghRCsgwV4IIVoBCfZCCNEKNJsxeyGak6Ki\nIpKSkrhy5Yq9myIEAO7u7nTo0AEXF5frqi/BXggbkpKS8Pb2JjIyEqWUvZsjWjmtNRkZGSQlJdGp\nU6fr+g4ZxhHChitXrhAQECCBXjQLSikCAgJu6EpTgr0QtZBAL5qTG/17bPHBPudyEa9++zPHL1yy\nd1OEEKLZavHBvkRr3tx0kqVbE+3dFCEazJ/+9Cd69erFqFGjuO2220hJSbFLGzZu3Gi3fQwdOpTj\nx48DsHr1ah566KEm2a9Zs2bNYsCAAYwePZopU6ZQUlJyQ9+3bNkyli1bdkPfcS0tPti39XTlnqgw\nvtidTM7lIns3R4gGs2DBAuLi4njooYd47bXX7N2cJjdu3Di+/fZbAL7//nvGjh1r5xbV9Nprr7Fx\n40b8/f3ZsGGDvZtzTQ4xG2dmbCSfJiTxacI5HhnZ2d7NEQ7mz2sOcTglt0G/s3eYD3+c0MdU2ays\nLNq0aUNeXh6TJk0iPz+frl27snTpUgBGjx7N4MGD2b9/P998843NbVevXmXWrFmkpKTQoUMHli5d\nyosvvkhRURGbN28mNzeX9evX4+bmxuTJkykpKUFrzejRo23WdXV1rdFOW+3705/+ZGoftowdO5aF\nCxfy2GOP8eOPP/Lf//3fpvcRGhpq8zhW3+/ly5eZMWMGFy9e5KabbuJf//qX6e+rLD09HU9PT9PH\nvri4uMZ+yxw6dIjHH3+c1atX4+3tbepvxIwW37MH6Nvel8GR/izfdoaSUm3v5gjRIF544QVGjRrF\n9u3befLJJ0lNTWX+/Pl89913JCYmcuHCBQC2b99OTExMeaC3te3tt9+mb9++bNq0iW7duvHuu+8C\ncOLECeLi4pg4cSI//PADixcvZvz48fz444/l87lrq1tdbe0zsw9bhg4dyt69e0lKSsLDw4OAgADT\n+7DF1n4XL15M3759iYuLIzU1lf3795v+PoD58+fTs2dPUlJSiImJMX3sa9tvamoqDz74ICtWrGjQ\nQA8O0rMHo3f/+Ed72HjsIrf1CrF3c4QDMdsDb2gLFixg2rRp5T9nZWXxzjvvsHTpUjIzMykoKACg\nb9++TJw4sUrd6tsOHz5c/vOwYcNYt24d/v7+zJgxA4CIiAgKCws5ffo0U6ZMASA6OrrWura4uLjY\nbJ+ZfdhisVgYNGgQr7zyCnfccUe99mGLrf0eO3aMrVu3snHjRrKzs0lOTjb9fWAM48TGxvK73/2O\nl19+meeee87UsS8sLLS539dff50BAwZw5syZOq8m6sshevYAY/uEEurjzjK5USsc1JIlS5g0aRIr\nVqwoHzIA8PLyqlG2+rY+ffqwfft2wOh59uljnMAqfw8Ywe3QoUMA7N2795p1zbbPzD5qM27cON58\n803GjRtXr33YYmu/PXr04Le//S0bN27k+eefJyIiwvT3lXFycsLf359Ll4wZgWaOfW37fe6551i0\naBHPPfec6f2bbmeDf6OduFiceHBoBJuPp3PiYp69myNEgxszZgwvvfQSt956K0B5b9CMRx55hEOH\nDjFq1CiOHz/OrFmzbJabPXs2n3/+OaNHjyY3N7dedc22z9Y+ajN27Fi8vLwYOnRovfZhdr+PPvoo\n69atY9SoUbz55puEh4eb/j4whnGGDx/Ot99+y7x582yWsXX8atuvu7s74eHh9OzZk9WrV9erLXVR\nWjePMe7o6Gh9o/ns0/OuEvvSD0wdEs5f7unbQC0TrdGRI0fo1auXvZshRBW2/i6VUgla69rHw6xM\n9eyVUkuUUtuUUs/WUe4NpdSE+tZrKIFeboyPasfnCUlcuiLTMIUQokydwV4pNRGwaK1jgM5KqW61\nlBsJhGqt19SnXkObFRtJfmEJnyUkNcXuhBCiRTDTsx8NrLS+3wCMqF5AKeUCvA0kKqXuMVuvMfTr\n4MeACD+WbztDqUzDFEIIwFyw9wTK7oJkArbmNc4ADgN/A4YopeabqaeUmq2UildKxaelpdW37bWa\nFRvJ6fR84o433HcKIURLZibY5wFtrO+9aqkzAFistT4PfADcYqae1nqx1jpaax0dFFTn4uim3dm3\nHUHebjINUwghrMwE+wQqhmCigEQbZU4AZXkKooEzJus1CldnYxrmxmNpnE7Pb6rdCtGgZs2aRf/+\n/YmOjubtt9+2d3NEC2cm2K8CpiulXgXuBw4ppZ6vVmYJcItSKg54DFhoo97ahmt2JTlJsPweOLWp\nyuYHhkbgYlEs35bYKLsVoim8/vrrfPPNN/z5z38uf6ReiOtRZ7oErXWuUmo0MAb4m3WoZl+1MpeA\nydXrVquX0xANrsEjEC4cgq2vQeebyzcHe7tz103t+Cw+id/f0QNPN4fJDCGa2rqn4fyBhv3O0Jvg\nzpdNFQ0ICOAXv/gFX3zxBU899RQ5OTlMmDCBZ555hkcffZR58+bRv39/5syZwyOPPEJQUBALFiwo\nT1ZWllpg5syZZGdnM2jQIP7xj38wc+ZMfvOb3zBs2DBmzZrF3Llz6dSpE7NmzaqyD+EYTM2z11pn\naa1XWgO9addbr15c3GHIHDjxLVw4XOWjmbGRXLpazL93yzRM0bIFBATw17/+lSlTprB161ZWrVpF\nRkYGkyZNKs9Vc/ToUQYPHgzAmjVrmDNnTnlmzBdffJGpU6eyefNmcnJyWL9+PTNmzODDDz+ksLCQ\nI0eOMGzYMF566aUa+xCOwTG6u4Mfhs3/C9v+BfdWpAodEO5Hvw6+LNuayLRhHWWZOXF9TPbAG1Nm\nZiYlJSUsWrSIZcuWkZ+fT0pKCrfddhuLFi3iyJEjVZKK3XHHHQwbNqz858OHDzN37lzAyCZ55MgR\nnnzySZ566inWrl3L3XffDRiJwbZt21ZlHwEBAU37y4pG4Ri5cTzawoBpsP8TuFRxEaGUYmZMJCfT\n8tlyIt2ODRTi+mVnZ7Nu3TrGjh3Lyy+/zMaNG3n66adp27Ytzs7OBAUFsWLFCiZNmlRex0wyLicn\nJ8aMGcPvf//78uyaPXr0qLEP4RgcI9gDxDwGpcWw460qm8dHtSPQy5X3ZBqmaIHmz5/PuHHjeOWV\nV1i2bBkLFy5k+PDhrF+/npAQ49GVX/7yl3z88cdVevLVPfPMM3z88ceMGDECPz+/8pTBkydPJiIi\ngo4dOwLw9NNP29yHaPkcKhEan0yH05vgd4fArSLx//9uOMbrP55g0+9vISLA4wZbKlqD1pAI7fvv\nv+cPf/gDL7zwQnkKYdG8NXoitBYj9gm4kgN7Pqiy+cGhHbEoxfvbE+3TLiGaodtuu42EhAQJ9K2E\nYwX78MEQEQPb3oCS4vLNob7ujO0byie7znG5sPgaXyBEheZy1SsE3Pjfo2MFe4DY+ZBzFo58WWXz\nrNhIcq8U88Ue84sdiNbL3d2djIwMCfiiWdBak5GRgbu7+3V/h2NMvays+53Qtgv89E/oMxGs0y2j\nO/rTu50P721N5IEhETINU1xThw4dSEpKoiET9AlxI9zd3enQocN113e8YO/kBLGPw1e/gzM/QaSR\nnkcpxazhkfzhs/1sO5VBbJdAOzdUNGcuLi506tTJ3s0QosE43jAOQNSvjDQKP/2zyua7o8Lw93CR\naZhCiFbHMYO9SxsYMhuOfwMXj5ZvdnexMHVIBN8evkBS1mU7NlAIIZqWYwZ7gMGPgLM7bHu9yuZp\nw4yHRz7YftYerRJCCLtw3GDvGQD9H7SmULhQvrm9Xxvu6B3Kx7vOcqWoxI4NFEKIpuO4wR4gZh6U\nFMHOxVU2z4yNJPtyEV/ulWmYQojWwbGDfUAX6PkL2PUOFFasWDWsc1t6hnqzbOsZmUcthGgVTAV7\npdQSpdQ2pdSztXzurJQ6q5TaaH3dZN2+t9K2MQ3ZcNOGPwlXsqukUFBKMTM2kiOpuexKzLJLs4QQ\noinVGeyVUhMBi9Y6BuislOpmo1g/YIXWerT1dUApFQAcrbTt2wZuuznhQyB8qJHrvlIKhXv7t8e3\njUzDFEK0DmZ69qOBldb3G6hYRLyyYcB4pdRO61WAMzAUGKKU2qqUWqWU8rZRr2nEzofsM3B0Tfmm\nNq4WpgwOZ/2h86RkF9itaUII0RTMBHtPoOxOZiZgK8H1LuB2rfUQwAW4CzgFjNVaxwL7gYeqV1JK\nzVZKxSul4hv1sfQed0HbzsZDVpXG6KcP60ip1ny440zj7VsIIZoBM8E+D2hjfe9VS539WutU6/t4\noBtGsD9RbVsVWuvFWutorXV0UFBQvRpeL04WY2ZOym44u618c3hbD27vFcKKnedkGqYQwqGZCfYJ\nVAzdRAGJNsq8r5SKUkpZgHuBfcALwATr55Os2+wn6gHwCICtr1XZPCs2ksz8Qr7an1pLRSGEaPnM\nBPtVwHSl1KvA/cAhpdTz1cr8BXgf2Ats01p/B7wKLFBKHQSuAu81XLOvg6sHDH4Ujn0NaT+Xb47t\nEkC3YC/e25oo0zCFEA6rzmCvtc7FuEm7HbhFa71Pa/1stTIHtdb9tNY3aa0XWLelaq2Haq37aq0f\n1VoXNcYvUC82UigopZgRG8mB5Bx2n822Y+OEEKLxmJpnr7XO0lqv1Fqfb+wGNSqvICMj5r6PIe9i\n+eaJA9rj7e7MMpmGKYRwUI79BK0tMfOgpBB2vl2+ydPNmcmDwll3IJULuVfs2DghhGgcrS/YB3Yz\npmLuehsKK9Icz4jpSInWfLhDsmEKIRxP6wv2AMOfgIIs2Pth+abIQE9u6RHMRzvOUlhcasfGCSFE\nw2udwT58KHQYbKRQKK2YXz8zNpL0vKt8fUCmYQohHEvrDPZKGSkUsk7D0a/KN4/sGkjnQE+5USuE\ncDitM9gD9BwP/pFVUig4OSlmxHRk77ls9p6TaZhCCMfReoO9kwViHofkeDi3o3zzfYM64OlqkWyY\nQgiH0nqDPRjLFrZpWyWFgre7C5Ojw/lqfwppl67asXFCCNFwWnewd/Uwnqo9uhbST5RvnhHTkaIS\nzYqdMg1TCOEYWnewBxjyKFhcq6RQ6BzkxajuQXyw/QxFJTINUwjR8kmw9wqGqKmwbwXkVeTUnxXb\nkYuXrrLuYMvOECGEECDB3hDzOBRfMRYmtxrdPZiOAR5yo1YI4RAk2AMEda+RQsHJSTF9WEcSzmRx\nMDnHzg0UQogbI8G+TOx8uJxhDOdYTY4Ox8PVIg9ZCSFaPAn2ZSJioP0g40atNYWCbxsXJg5sz+p9\nKWTkyTRMIUTLJcG+TFkKhcxTxmpWVjNjIiksLuXjXefs2DghhLgxpoK9UmqJUmqbUurZWj53Vkqd\nVUpttL5usm7/s1Jql1LqXw3Z6EbTcwL4dazykFW3EG+Gdw3gg+1nKJZpmEKIFqrOYK+UmghYtNYx\nQGelVDcbxfoBK7TWo62vA0qpQRgLlQ8BLiqlbm/QljcGi7MxM+fcDjhbkUJhZkwkqTlX2HD4gh0b\nJ4QQ189Mz340sNL6fgNGAK9uGDBeKbXTehXgDNwMfK6NVby/AUZWr6SUmq2UildKxaelpVX/2D4G\nPAjufrCtond/W68QOvi3kRu1QogWy0yw9wSSre8zgRAbZXYBt2uthwAuwF1m6mmtF2uto7XW0UFB\nQfVte+Nw9TRSKBz5CjJOAmCxZsPceTqTI6m5dm6gEELUn5lgnwe0sb73qqXOfq112Yof8UA3k/Wa\npyGzweJiLG5idX90OO4uTvKQlRCiRTITgBOoGLqJAhJtlHlfKRWllLIA9wL7TNZrnrxDoN8UY9nC\n/HQA/Dxc+eWA9qzam0xWfqGdGyiEEPVjJtivAqYrpV4F7gcOKaWer1bmL8D7wF5gm9b6O2ALMEAp\n9X/A08AKWpLY+dYUCkvKN82MjeRKUSmfxMs0TCFEy1JnsNda52LcpN0O3KK13qe1frZamYNa635a\n65u01gus20qB24HNwJ1a69MN3vrGFNQDuo+DnYuhqACAnqE+DO3Ulve3naGkVNu5gUIIYZ6pcXSt\ndZbWeqXWul4pILXWBVrrz7TWp66veXYWOx8up8O+j8s3PTQ8kuTsAr47ItMwhRAtR8u5aWoPHYdD\n2ABrCgXjgarbe4UQ5usuN2qFEC2KBPtrKUuhkHECfl4HgLPFiWkxHdl6MoOfL1yycwOFEMIcCfZ1\n6XUP+EVUSaEwdXAErs4yDVMI0XJIsK+LxRmGzYOz2+DcLgDaerpyT1QY/96dTM7lIjs3UAgh6ibB\n3owB08Ddt0oKhZmxkRQUlfBpgkzDFEI0fxLszXDzguiH4cgaIwUy0Le9L4Mj/Vku0zCFEC2ABHuz\nhs4BZYHti8o3zYyN5GzmZTYeu2jHhgkhRN0k2JvlHWqkUNjzAVzOBGBsn1BCfdwlG6YQotmTYF8f\nsY9D0eXyFAouFiceHBrB5uPpnLiYZ+fGCSFE7STY10dwL+h2B+x8C4quAPCroRG4WpxYvi3Rrk0T\nQohrkWBfX7HzIT8N9n8CQKCXG+P7tePzhCQuXZFpmEKI5kmCfX1FjoR2UVVSKMyMjSS/sITPEpLs\n3DghhLBNgn19KQWxT0D6z3D8GwCiwv0YEOHH8m1nKJVpmEKIZkiC/fXofS/4hldJoTArNpLT6fks\n2nTSjg0TQgjbJNhfD4szDHsMzvwESQkATOgXxt1RYfzPN8dYsfOsnRsohBBVmQr2SqklSqltSqln\n6ygXopTaY33vrJQ6q5TaaH3d1BANbjYGTge3ihQKTk6KhZOjGN0jiAVfHGDdgdQ6vkAIIZpOncFe\nKTURsGitY4DOSqlu1yi+kIpFxvsBK7TWo62vAzfe3GbEzRuiH4LDX0JWIgCuzk4senAQAyL8efLj\nvWw5nm7fNgohhJWZnv1oYKX1/QYqFhGvQil1K5APlK1mNQwYr5Taab0ycL7BtjY/Q+fWSKHQxtXC\nuzMH0ynQk9nvx7P3XLYdGyiEEAYzwd4TSLa+zwRCqhdQSrkCz2EsLF5mF3C71noI4ALcZaPebKVU\nvFIqPi0trb5ttz+fdtDvfti9vDyFAoCvhwvvPzyEAC9XHlq6kxMXZZETIYR9mQn2eVQMzXjVUudp\n4A2tdeVu7H6tddnAdTxQY/hHa71Yax2ttY4OCgqqR7ObkRhrCoX4d6tsDvZx54OHh2JxcmL6kp0k\nZxfYqYFCCGEu2CdQMXQTBSTaKHM7ME8ptRHor5R6B3hfKRWllLIA9wL7bry5zVBIb+h6O+x4C4qv\nVvmoY4Any389hLyrxUxfsoOMvKu1fIkQQjQuM8F+FTBdKfUqcD9wSCn1fOUCWutRZTdigb1a60eA\nvwDvA3uBbVrr7xq26c1I7HzIvwj7V9b4qHeYD+/OGkxyVgGzlu6SlApCCLtQWtf9xKdSyh8YA8Rp\nrc/XVf56REdH6/j4+Mb46sanNbw1EooL4bHt4FTzHPrD0Qs8ujyBIZFtWfrQYNxdLHZoqBDC0Sil\nErTW0XWVMzXPXmudpbVe2ViBvsVTCmKfhPRjsOllm0Vu7RnC/06OYtupDJ5YsYfiktImbqQQojWT\nJ2gbSt/7oP802PQKxC20WeTeAe3544TebDh8gf/64gBmrqqEEKIhON7cd3txcoK7/wmlRfDDX8Hi\nCsOfqFHsoeGdyLpcxD+/P46/pyvP3NnLDo0VQrQ2EuwbkpMF7nkDSorg2+fA4gLDflOj2O9u70ZW\nfiFvbTqFv4crc2/uYofGCiFaEwn2Dc3iDBMXGz389U+DkzMMebRKEaUUf767D9kFRby87ih+bVyY\nOiTCTg0WQrQGMmbfGCwucN+70P1O+Pr3kPBejSJOTor/nRzFzd2D+K8vDrD+oCROE0I0Hgn2jcXZ\nFe5/D7qOgTVPwt6PahRxdXZi0bSB9A/344kVe9l6QhKnCSEahwT7xuTsBlM+gM6jYdVjsP/TGkU8\nXJ15d5aROO3R5fHsT5LEaUKIhifBvrG5uMPUjyByBHwxBw59UaOIn4cryx8egr+nK7OW7uLExTw7\nNFQI4cgk2DcFVw/41ccQPgQ+fwSOfFWjSIg1cZqTUkxfskMSpwkhGpQE+6bi5gUPfgphA+DTWXBs\nfY0ikYGSOE0I0Tgk2DclN2948DMI7Qsrp8OJmrnheof5sGSmkTjtoWW7yLtabIeGCiEcjQT7ptbG\nD6b9G4J6wMcPwqmNNYoM6dSWNx4cyKGUXGYvj+dqcUnTt1MI4VAk2NuDR1uY/iW07QIfTYXELTWK\n3NYrhIWT+7H1ZAZPrtgridOEEDdEgr29eAbAjC/BLwI+vB/Obq9R5JcDOvDHCb1Zf+g8C744KInT\nhBDXTYK9PXkFwczVxlq2H0yCpIQaRR4a3oknbu3KJ/HneGX9MTs0UgjhCEwFe6XUEqXUNqXUs3WU\nC1FK7alvvVbNOxRmrgHPQHj/l5Cyt0aR343pzrRhEby56SRvbTpph0YKIVq6OoO9UmoiYNFaxwCd\nlVI1Fg6vZCHWxcnrWa918wkzAn4bX1h+D5w/UOVjI3FaX8b3a8dL646yctc5OzVUCNFSmenZjwbK\nFlfdQMXi41UopW4F8oGy1axM1RNWfuFGwHf1NAL+hcNVPrY4KV69vz+jugfx9L/3880hWTRMCGGe\nmWDvCSRb32cCIdULKKVcgeeAp+tZb7ZSKl4pFZ+Wllafdjsm/0gj4Du5wPK7Ie3nKh+7Ojvx5rSB\nRIX7Mf+jPWw9KYnThBDmmAn2eViHZgCvWuo8Dbyhta6cxavOelrrxVrraK11dFBQkPlWO7KALjDr\nK0DBexMgo+oYvYerM0tnDSYy0INH35PEaUIIc8wE+wQqhmCigEQbZW4H5imlNgL9lVLvmKwnbAns\nZvTwS4uNgJ95usrHfh6uLP/10PLEaSfTJHGaEOLazAT7VcB0pdSrwP3AIaXU85ULaK1Haa1Ha61H\nA3u11o/YqLe2YZvu4IJ7GvPwiy7De3dD9tkqH4f6uvP+w0NxUjD9nR2kSOI0IcQ11Bnstda5GDdb\ntwO3aK33aa1rnUppDfi26uU0QHtbl9C+MH0VXM0xevg5yVU+7hToyXu/HsKlK0bitMz8Qjs1VAjR\n3JmaZ6+1ztJar9Ra12sKyPXWE5WE9YfpX8DlTCPg51ZdvrBPmC/vzIwmKauAh5bulMRpQgib5Ana\nlqD9ICNbZt4FY5ZO3sUqHw/tHMC/HhjIwZRc5rwvidOEEDVJsG8pIobCAyshJ8kYw8+vOu3y9t4h\n/O2+fvx0IoMnVuwhX3r4QohKJNi3JJHDjRWvsk7D8nuNoZ1K7htkJE7bcPgC4/4vThYwF0KUk2Df\n0nS+2VjTNv1nI5dOQdV59g8N78TKOTE4OznxwDs7eHbVARnHF0JIsG+Rut4GUz6AC4fgg/vgSm6V\njwdHtuXrJ0byyIhOfLjjLGP/HseW49LLF6I1k2DfUnW/A+5/D1L3woeT4WrVB6vauFp4dnxvPpsb\ng5uLE9OW7OCZf+/n0pUiOzVYCGFPEuxbsp6/gEnvQtIu+GgKFObXKDKoo9HLn3NzZz7ZdY6xf49j\n08+Sh0iI1kaCfUvX+x6YuBjOboUVv4Kimk/SurtYeObOXnz+m1g83JyZ+e5O/t+n+8gpkF6+EK2F\nBHtHcNMkuOcNOB1nLGJedMVmsQER/nw1fwSPje7Cv/ckc8ffN/HD0QtN3FghhD1IsHcU/X8Fd/8T\nTn4Pn0yrMS2zjLuLhT+M68kXj8Xi18aVXy+L5z9W7iXnsvTyhXBkEuwdycAZMP4fcPIHeD0a9n0M\ntSxS3q+DH6vnD2f+rV35cm8KY/6+iW8PSy9fCEclwd7RRD8Ec+LAvxN8McdY9SrD9rq1bs4W/vOO\nHnw5bzhtPV15dHk8v/14D1mSUE0IhyPB3hGF9oWHN8BdCyFlD7wRA3H/A8W2g3jf9r6sfnwEv729\nG1/tT2XM3+NYfzDVZlkhRMskwd5ROVlgyKMwbyf0uBN+eB7eGglnttks7ursxG9v787qx0cQ7O3G\n3A928/hHu8nIu9rEDRdCNAYJ9o7Op53x8NWvPjHm4S8dB2uehIIsm8V7h/nw5ePD+c8x3fnm0Hnu\n+Hsca/dLL1+Ilq7Bgr1Sqq1SaoxSKrChvlM0oB7j4LHtEPM47F4Orw+BA5/ZvIHrYnFi/m3dWDN/\nBGF+bZj30W4e+zCBdOnlC9FimQr2SqklSqltSimbK1QppfyBr4AhwI9KqSCllLNS6qxSaqP1dVMD\ntltcDzcvGPsCzN4Ivu3h84fhw0mQlWizeM9QH754LJb/N7YH3x2+yJhXN7F6Xwq6lhk+Qojmq85g\nr5SaCFi01jFAZ6VUNxvF+gH/obV+AfgGGGjdtqJsbVqt9YGGbLi4Ae2i4JHvYdwrcHY7/GsYbPkH\nlNSca+9scWLeLV1Z+8QIIgI8eWLFHua8n8DFS7Yf3BJCNE9mevajgZXW9xuAEdULaK03aa23K6VG\nYfTutwHDgPFKqZ3WKwPn6vWUUrOVUvFKqfi0NMnX0qScLDBsLszbAV1uhe/+CItHw7ldNot3C/Hm\n87kxPH1nTzb+nMaYV+NYtSdZevlCtBBmgr0nULbSdSYQYquQUkoBU4AsoAjYBdyutR4CuAB3Va+j\ntV6stY7WWkcHBQVdR/PFDfPtAL/6CKZ8aDx1u2QMrP1PuFJzfXhnixNzb+7C10+MpHOQJ7/9ZC+P\nLo/nQq708oVo7swE+zygjfVONOUqAAAXIElEQVS9V211tGEesB+4G9ivtS6bxhEP2Br+Ec1Fr/FG\nL3/oHNi1xLiBe2iVzRu4XYO9+GxuLM/+ohebj6cz5tVNfJaQJL18IZoxM8E+gYqhmyggsXoBpdRT\nSqkZ1h/9gGzgfaVUlFLKAtwL7Lvx5opG5e4Dd74Cj34PXkHw6UxYMRWyz9UoanFSPDKyM+ueHEn3\nEG9+/+k+fr1sF6k5NbNuCiHsT9XVG1NK+QCbge+BO4GpwGSt9bOVyvhjjOu7AQeBeUAf4CNAAau1\n1guutZ/o6GgdHx9//b+JaFglxbBjEfz4IqDglv+CoXPBUuPWCyWlmve2JvK3b47i4uTEs+N7cX90\nOMbInhCiMSmlErTW0XWWM3PpbQ3mY4A4rfX5BmhfDRLsm6nss7D293D8G2MWz4T/g7ABNosmpufz\nh8/3s/N0JiO7BfLyff1o79fGZlkhRMNo0GDfFCTYN2Naw+FVsO4pyE+DIXPg1gXg5l2jaGmp5v3t\nZ3hl/VGKSzT3DWrPIyM70yXIyw4NF8LxSbAXDe9KDnz3Z4h/F3zC4K7/MZZGtOFc5mXe3HSSTxOS\nKCopZUyvEObc3IVBHf2buNFCODYJ9qLxnNtp5Ne5eBh6jjeCvk+YzaJpl66yfFsiy7edIaegiMGR\n/swZ1YVbewbj5CRj+kLcKAn2onGVFMG212HjK+DkDLc9B4MfMR7WsiH/ajEr48/xzubTJGcX0CXI\nkzmjunDPgDDcnG3XEULUTYK9aBqZp2HtfxirY4UNNG7gtutXa/GiklK+PpDKW5tOcTg1l2BvN349\nohMPDI3Ax92lCRsuhGOQYC+ajtZw8HNY/7TxFG7MYzD6GXD1vEYVzZYT6by16RRbTqTj5ebMA0Mj\neGh4JO18ZQaPEGZJsBdN73ImfPcn2P0e+EbAuBehxy/A6drP7h1MzuGtuFOs3Z+CxUlxT//2zB7V\nme4hNWf7CCGqkmAv7OfMVljzW0g/Bm27GA9j9X/ASLF8DecyL7Nky2k+2XWOgqISbu0ZzJxRnRnS\nqa08oCVELSTYC/sqKYIjq2H7IkjaBW6+MHA6DJkN/h2vWTUrv5D3t59h2dZEMvML6R/ux5xRnbmj\nTygWmcEjRBUS7EXzkRRvBP3Dq0CXGnPzhz0GETFwjR57QWEJn+1O4u24U5zNvExkgAePjurMfQM7\n4O4iM3iEAAn2ojnKSYZd70DCUmMN3NB+RtDvOxGc3WqtVlKq+ebQed7cdJL9STkEerkyKzaSacM6\n4ufh2oS/gBDNjwR70XwVXoYDK43eftpR8Aw25uhH/9rItlkLrTXbT2XyVtxJNh5Lw8PVwpTB4Tw8\nohMd/D2a8BcQovmQYC+aP63h1I9G0D++ASyucNNk44buNebqAxw9n8viuFOs3puCBib0a8fsUV3o\nHebTNG0XopmQYC9alvTjsOMt2PsRFOVD5Egj6Pe4s9ancgFSsgtY+tNpPtpxlvzCEkZ2C2TuzV2I\n7RIgM3hEqyDBXrRMBdmw533YsRhyzoJfRyPoD5hmLK5Si5yCIj7ccYalPyWSdukqfdv7MHtUF+7q\nG4qzxcwaPUK0TE0e7JVSbYFBwB6tdXp960uwF1WUFMOxtcYQz9lt4OplBPwhsyGgS63VrhaXsGpP\nMm/FneJUWj4d/NswbVhHJkSFSW594ZAaevGSJUBvYK3W+nkbn/sDa62vqcCtWuu0uupVJsFe1Cpl\nD2x/00jJUFpsDO0MnQudRtU6dbO0VPPdkQu8vfkUuxKzAIju6M+EqDDuvCmUYG/3pvwNhGg0DRbs\nlVITgbu11rOUUu8CL2mtj1crczNwVWu9XSm1EPgW8KyrXmUS7EWdLp03FkOPfxcup0NwHxj2G+Om\nrkvtwftMRj5f7U9lzb4Ujp6/hJOCmC4BTOgXxri+oTJ9U7RoDRns/wms11p/rZSaCrTRWi+tpewo\n4HlgvPVfU/VAgr2oh6IrcPAzY4jnwkHwCDCmbQ5+BLxDr1n15wuX+GpfCqv3pZCYcRlnJ8Wo7kFM\niGrHmN6heLnVXGNXiObMbLA385ftCSRb32cCA2vZoQKmAFlAkZl6SqnZwGyAiIgIE00RAqMXP2Aa\n9H8QErcYQT9uIWz5h/GA1tC50N7mnyndQ7z5jzt68Lsx3TmUksuafSms2ZfCD0cv4uZ8gFt7BjMh\nKoxbewbLU7rCoZgJ9nlA2Z0tL8Dm1AZtXCLMU0r9FbjbTD2t9WJgMRg9+3q1XAiloNNI45V5ypjB\ns+cD2P8JhA8zhnh6jgdLzT9zpRR92/vSt70vT43ryZ5zWazZl8pX+1NZd/A8nq4WxvQOYUJUGCO7\nBeHqLDN6RMtmZhhnBhCstV6olPozcExr/VG1Mk8BqVrr5Uqp14CvgJC66lUmwziiQVzJhb0fwo43\nISsRfMNhwHTocy8E9aizenFJKTtOZ7JmXwrrDp4np6AI3zYujOsTyoSoMIZ1bitTOUWz0pBj9j7A\nZuB74E6M2TaTtdbPVirjD6wE3ICDwDzAu1q9YVrrnNr2I8FeNKjSEvh5vRH0T28GNAT1hF53Q+97\nIKTPNZOwARQWl7LlRBpr9qWy4dB58gtLCPRy5Rc3tWNCVBgDI/xlHV1hdw099dIfGAPEaa3P16MR\nputJsBeNJjcVjn4Fh7+EMz8ZmTfbdoHe1sDfrn+dgf9KUQk/Hr3Imv0pfH/kIleLSwnzdWd8VBgT\n+oXRt72PPLEr7EKeoBXClry0isB/Og50CfhFWHv890L7QXWurJV3tZjvDl9g9b4U4n5Oo7hUExng\nwYSoMCZEhckKW6JJSbAXoi6XM+HY10bgP/kjlBaBT3voNcHo8YcPvWZeHoDsy4WsP3ieNftT2HYy\ng1INPUK8mRDVjvH9wogMrH0dXiEaggR7IeqjIBt+/sYI/Ce+g5Kr4BViBP5ed0PH4TZn9VR28dIV\n1h04z5p9KcSfMZ7a7dfBlwn9wvhFv3aESboG0Qgk2Atxva5eMlIuH/4Sjn8LRZeNB7d6jjfG+Tvd\nDBaXa35FcnYBa/ensGZfKgeSjXkJfdv7MKJrEKO6BTIo0h83Z5nHL26cBHshGkLhZaOnf/hLo+df\neAnc/YylFXvdDV1uueYqWwCn0/NZuz+FuJ/T2X02i+JSjbuLE0M6BTCyayAjugXSM9RbbvCK6yLB\nXoiGVnTFWGzl8Jdw9Gu4mgNuPtB9nDHG3/U2cLn2UE3e1WK2n8xgy4l0Nh9P42RaPgBB3m6M6BrI\niK6BjOwWSLCPJGoT5kiwF6IxFRcas3kOr4Kja6EgE1w8ofsd1sA/Bty86vyalOwCthxPZ/OJdH46\nkU5mfiFg3OQd0c3o9Q/t1BYPV8nZI2yTYC9EUykphjNbjB7/ka8g/yI4u0PX243pnN3HXnPhlTKl\npZrDqblsPp7OlhNp7ErMorC4FFeLE4M6+jOim9Hr7xPmi0Ue5hJWEuyFsIfSEji73Rr418ClFGNt\n3S63Qrcxxs3dgK51PsQFUFBYwq7ETLacSCfu5zSOnr8EgJ+HC8O7GIF/RLdAWWy9lZNgL4S9lZZC\ncrw18K+G7LPGdu92xsIrZS8/cxlf0y5d5acT6eU9/wu5VwHoFOhZPtYf0yUAb/drzxQSjkWCvRDN\nidaQddoY5y975acZn/lHGgusd7rZyOBZR05+4+s0xy/mGYH/eBo7TmdyubAEi5Oif7gfI7oGMqp7\nIFEd/CRxm4OTYC9Ec6Y1pB2tCPyJm+GKNU9gYI+KXn/kCPBoW+fXFRaXsvtslnGz93ga+5Nz0Bq8\n3ZwZ1iXAGPLpGkinQE+Z4ulgJNgL0ZKUlsD5AxXB/8xWKMoHFIT2tfb6R0FEjKmbvdmXC9l6MoPN\n1uCflFUAQHu/NkRH+jMwwp8BEX70aueDi/T8WzQJ9kK0ZCVFkLzb2uuPg7M7jBQOymKswtVplDH0\nEz4UXK99g1ZrzZmMy2w+kc7WE8aDXWXj/W7OTvTr4MuACH8GhPsxIMKfUF+Z49+SSLAXwpEUXYGk\nnRU9/+QEKC02Zvp0GFIx7NN+EDhfewF1rTWpOVfYczab3Wez2HM2i4PJuRSWlAIQ5utuBP8II/j3\nCfORJRqbMQn2Qjiyq5eMKZ6nNxmLs6TuAzS4eEDEsIrg365/nZk7Aa4Wl3Ak9RK7z2Sx51w2e85m\nlQ/9uFgUvcN8GWgN/gPC/ejg30bG/puJhl68ZAnQG1irtX7exue+wMeABcjHWHi8FDhlfQHM11of\nqG0fEuyFuAEFWZD4U0XPP+2Isd3NFyKHVwz7BPeuM19/mYu5V6yB3wj++5NyKCgqAYz0DmXDPgMi\n/OjXwVee8rWThlyWcCJwt9Z6llLqXeAlrfXxamUeA45rrb9VSi0C1gFJwBSt9VNmGizBXogGlHex\nYpbP6ThjQXYwsnd2GAxhA40hn/YDTc32AWN93qPnLxknAOsVwOl0I7ePxUnRM9TbGPoJ92dgR38i\nAzyk998EGjLY/xNYr7X+Wik1FWijtV56jfKfAQuBgRhr0eYDB4A5Wuvi2upJsBeiEWWfMwJ/4hZj\nvD/tGGD9v+8faQT+shNAu6g6b/qWycwvZN+5srH/bPaeyybvqvHf3M/DhQHhftaZP/5EhfvKA1+N\nwGywN3Pd5QkkW99nYgTx2nYaA/hrrbcrpUqA27XWqUqp5cBdwOpq5WcDswEiIsw9RSiEuA5+4dD/\nAeMFcCUXUvcaM36SE+DcTjj4ufGZskBwL6PXX3YCCO5lM4d/W09XbukZzC09gwEoKdWcuJjHHmvw\n33Mui40/p6G1kSGiW7AXAyP86R/uR+8wH7qHeMvN3yZipmf/f8AKawCfCPTUWr9oo1xbYANwn9b6\njFLKTWt91frZE4CL1vp/a9uP9OyFsLNLFyBld8UJIGW3cS8AwLkNtOtX6QpgILTtbCrHT+6VIvZZ\nx/7LrgByCooAcFLQOciLXu186NXOm17tfOjdzodgbzcZAjKpIYdxZgDBWuuFSqk/A8e01h9VK+OK\nMU7/stb6W+u2lcALwEHgW+BFrfV3te1Hgr0QzUxZiofkSieA1H1QbMzSwd3PCPqVh4C8Q0x8rTHv\n/0hqLkdSczmceokjqbkkZxeUl2nr6WoE/1Af64nAh67BXrg6ywNg1TVksPcBNgPfA3cCU4HJWutn\nK5X5DfAisM+6aRFwCPgIUMBqrfWCa+1Hgr0QLUBJsTHTJzmh4iRw8TBoY5YOPu2rngDCBph64hcg\np6CIo9YTwJHUSxw5n8ux85e4WmzM/3exKLoEedG7XcUJoFc7bwK8rr1SmKNr6KmX/sAYIE5rfb4B\n2leDBHshWqjCy3B+f6UTQIJxRQCAgsDuVU8AoX3rXMqxTHFJKYkZ+eW9/7JX2RPAAMHeblWCf+92\nPnQK9Gw1CeDkoSohhP1czrSO/++xngQSjEVdAJxcIKQPhPSFkN7G3P+QPuAVbPrrM/MLKw0DGVcC\nJy5eoqjEiGduzk50D/Euvw/Qq50PvUJ98PVwvNlAEuyFEM2H1pCbXNH7T9ljDP+UpXkG8Ag0Zv2E\n9Kk4AQT1NLW8IxiZP0+m5VW6AjCuBjKsSz2CkQiu8gmgR6g3Hdt6tOirAAn2QojmLy/NCPoXD8OF\nQ9b3R60ZP638OlY6AfSG4D4Q0MXmVNDqtNakXbpa3vsvOxGcSs+npNSIfS4WRccAT7oEedIlyMt4\nBXvROcgTnxbwXIAEeyFEy1RaCtlnrCeAw3DxkPFvxomKG8EWV+NeQOUTQHAv8O1gajrolaISjl/I\n4+h5I/CfvJjHybQ8zmRcpri0IiYGe7tZg3/VE0E7H3ecmsk6wBLshRCOpfgqpP9c9QRw8QjkJlWU\ncfO1DgVZ7wWUnQza+JvaRVFJKWczL1uDfz4n04yTwImLeVy6UpEAoI2Lhc5VrgSM950CPZv8ITEJ\n9kKI1qEg2wj6lU8AFw9VrPwF4B1W8wQQ2ANczOXu11qTnldYHviNk4BxRVD5+QClINzfo2JIKNh6\nMgjypK2na6M8KCbBXgjRemkNuSmV7gVYTwBpx6DEesNWWaBtJwjoZtwDCOgKgd2Mf71CTA0HARQU\nlnAqvSL4l50ITqXllT8jAEauoLLAX3lIKNy/zQ3dIJZgL4QQ1ZUUQ+bJihNA2hHIOGVsK75SUc7V\nq+IEENC10gmhC7j7mtpVaakmObug4iogLa98eCg9r+I5AReLYmS3IN6dNfi6fqWGTIQmhBCOweIM\nQT2MV2WlpcbYf8YJyDhp/fcEJMXDwX9TniEUwDPYehXQtdLJoKuRPbTSw2JOTorwth6Et/VgdLXd\n5Vwu4mR6RfD3dm/8UCw9eyGEuJaiK5CVWHECyDhecUKo/JyAcgK/COtVQNeqVwY+7U0vGlNf0rMX\nQoiG4OIOwT2NV3UF2cYQUMZJSD9ecUI4s7XqswLObSqGgQKqXRGYXDzmRkmwF0KI69XGz7ri16Cq\n27WGS+etVwGVhoYuHIKja43F4su/oy10uQUmvduoTZVgL4QQDU0p8GlnvDqNqvpZSRFknak0LHTC\n9HMAN0KCvRBCNCWLi3FzN7Brk+625Wb/EUIIYZoEeyGEaAUk2AshRCtgKtgrpZYopbYppZ6t5XNf\npdQ6pdQGpdQX1jVp66wnhBCiadQZ7JVSEwGL1joG6KyU6maj2IPAq1rrO4DzwDiT9YQQQjQBMz37\n0cBK6/sNwIjqBbTWb2itv7X+GARcNFNPKTVbKRWvlIpPS0ur/rEQQogGYibYewLJ1veZQEhtBZVS\nMYC/1nq7mXpa68Va62itdXRQUFC9Gi6EEMI8M/Ps84A21vde1HKCUEq1BV4D7qtPPSGEEI3PTLBP\nwBiC2Q5EAceqF7DekP0UeEZrfcZsvSo7SUhIV0qduVaZFiAQSLd3I5oROR5VyfGoIMeiqhs5Hh3N\nFKoz66VSygfYDHwP3AlMBSZrrZ+tVOY3wIvAPuumRcC6avWGaa0rLR3jeJRS8Wayz7UWcjyqkuNR\nQY5FVU1xPOrs2Wutc5VSo4ExwN+01uepCOplZRZhBPgqqtVz6EAvhBDNmancOFrrLCpm1ph2vfWE\nEEI0LLlp2rAW27sBzYwcj6rkeFSQY1FVox+PZrNSlRBCiMYjPXshhGgFJNgLIUQrIMHeJFvJ3mwl\nejO7zREopUKUUnus71v1sQBQSr2hlJpgfd9qj4dSyl8p9bU1Fcpb1m2t7nhY/39str53UUqtUUr9\npJT69Y1uux4S7M2rnuxtKtUSvdlK/ubgCeEWAm3M/t6OfCyUUiOBUK31GjkeTAc+tM4b91ZK/YFW\ndjyUUv7AexhpYwDmAwla6+HAJKWU9w1uqzcJ9ibZSPY2jZqJ3kab3NbiKaVuBfIxTnyjad3HwgV4\nG0hUSt1DKz8eQAbQVynlB4QDnWh9x6MEmALkWn8eTcXvFgdE3+C2epNgX09lyd6Ac9RM9GYr+Zvp\nRHIthTU9xnPA09ZNZn9vhzsWVjOAw8DfgCHAPFr38diC8Qj/E8ARwJVWdjy01rnVHiS9kf8jDXJc\nJNjXQ6Vkb7/GdqI3s9tauqeBN7TW2dafW/OxABgALLY+Xf4BRu+rNR+PPwJztdZ/AY4CD9C6jwfc\n2P+RBjkujnQwG5WNZG9lid7ASPSWWI9tLd3twDyl1EagPzCB1nssAE4Ana3vo4FIWvfx8AduUkpZ\ngKHAy7Tu4wE3Fi8a5LiYSpcgAHgYGAgsUEotAJYC05VSYVgTvQEa2GxiW4umtR5V9t4a8O/G3O/t\ncMfCagnwrlJqKuCCMca6uhUfj5cw/n90BLYBf6d1/32AcbP2a+uN/N7ADoyhmevdVm/yBO0NsN5x\nHwPEWS/hTW9zNHIsqpLjUZUcD7CexEYA35SN59/ItnrvX4K9EEI4PhmzF0KIVkCCvRBCtAIS7IUQ\nohWQYC+EEK2ABHshhGgF/j/TLBFmuNQruQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2148437b748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x,y4,label=\"Parrondo and Van den Broek\")\n",
    "plt.plot(x,y5,label=\"Devroye\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此n=10000时，Devroye给出的上界最小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 5\n",
    "和上题一致，我们依旧作图看一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:47: RuntimeWarning: invalid value encountered in double_scalars\n",
      "D:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:45: RuntimeWarning: divide by zero encountered in long_scalars\n",
      "D:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:46: RuntimeWarning: divide by zero encountered in true_divide\n",
      "D:\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:46: RuntimeWarning: divide by zero encountered in long_scalars\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAD6CAYAAABXh3cLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnXd4VFX6xz83yUwmvfceEhIg9EAg\ntNAERFERxFVUXBTLLq5bbLsW1l1FXHULFsSGhdWfZVEsoPQehCA1CZAA6b2Qnkxm7u+PO5kUElKH\nSTmf55nnztx7zj3vpHznnfe85z2SLMsIBAKBoG9iYW4DBAKBQNB1hIgLBAJBH0aIuEAgEPRhhIgL\nBAJBH0aIuEAgEPRhhIgLBAJBH0aIuEAgEPRhhIgLBAJBH0aIuEAgEPRhrEw9gLu7uxwcHGzqYQQC\ngaBfkZCQUCjLskd77Uwu4sHBwRw9etTUwwgEAkG/QpKktI6061A4RZIkL0mS9hmeh0qStEOSpOOS\nJD3THSMFAoFA0D3aFXFJklyADwE7w6nfAs/KsjwKmCNJUrvuvkAgEAhMQ0c8cR2wBCgzvC4CRkiS\n5AVYA6Umsk0gEAgE7dBuTFyW5TIASZIaTm0FHgH8gZ1AvamMEwgEAsHV6UqK4ZPAMlmW/wLYALNb\nNpAkaYUkSUclSTpaUFDQXRsFAoFA0AZdEfEQIECSJA0wBrhiVwlZltfLshwty3K0h4cImQsEAoGp\n6IqIPwfsBgqADJSQikAgEAjMQIfzxGVZjjMcvwe+N5VBDSSklXAotZBRAS4M93fCyUZl6iEFAoGg\nz2HyxT5d5cilYl756Zzx9SAPO0YGODM6wJmRAc5EejuithJVAwQCwcBGMvVGydHR0XJXV2xertZy\nKvMyxzNKOJ5xmeMZpRRW1AKgtrJgmK8jowKcjY9AV9umWTQCgUDQZ5EkKUGW5eh22/VmEW+JLMtk\nX67heHopJzJLOZ5eyqmsy1RrdQC42KoYGeDMSH9nRgUqR1c7dY+MbRb0eqjIg8sZykPjDP7jQONo\nbssEAoGJ6aiI99pwSmtIkoSfsw1+zjbMH+EDQL1Oz7m8CqOon8gsZe+58+gNn01BbraKqBvCMMN8\nHdGoLM34LppQXwdlmVBqEOnLDc/TlWNZFujqWnSSwGsYBMRA4AQIGA/OQSC+gQgEA5I+5Yl3lMra\nek5lKeGXExmlHM8oJedyDQBWFhJDfByNoj4qwJlQdzssLEwggrXljQJdmq6I9OWMxnPluVyRoWnv\nDc4B4BTQeHQKACd/qMyHjJ8hPR4yj0JdeWOfwBgImKCIu88IsBQTwQJBX6ZfhlO6Q15ZTTNRP5l5\nmYpaZbGpg8aKkf7OjAxwYlSACyMDnPB00Fz9hrIMlYWNXnNrnnRNi4oEFipw8jMIdGATofZvFGor\n6469Ib0O8s5AxmHlkX5YGRfAygb8xhqEPUbx1m1cOvkTEwgE5kSIeDvo9TKpBRUcN4j6icxSknPK\nqTfEYfycbRgZ4MQELz2z1KfxlQsbY9OlBsGur25+U7V9cw/aeAxUBNreGyxMmFFTlt0o6BmHIfck\n6A1VETwiDYJuCMO4hooQjEDQixEi3gVqtDrOZF/ml/RSThiyYh4rf5kFlocA0Nm6Y9lMmFsItsa5\ndwljXSVkHYOMeCUMk3EYai4r1+w8Gr30gAngO6rj3wIEAoHJESLeQ+heHUaq1SBuLViOVlLz4LRB\nrJgaiq26T80JK+j1UHi2ibceD8UXlGuWavAd3WTCNAbs3M1rr0AwgBEi3hNU5MMr4XDdC2RE/pqX\ntibz/ckcvB01PD43gptH+ZlmQvRaUmGYLM2IV4Q953hjRozroEZBD5wAbuGmDQcJBAIjQsR7grNb\n4dMlcO9WCJoIwNFLxfztu0ROZF5mhL8Tz94wlOhgVzMb2oNoaxQhT49vFPeqIuWajQv4j4eQKTBo\nJngO6V3hI4GgHyFEvCfY+QLsexWeygC1nfG0Xi/zzYks1mw5S25ZDfOH+/DkvEgCXG3NaKyJkGUo\nSjXE1Q8r4l5oKIdg7w2DZhge00X4RSDoQYSI9wQfL1TCDQ/tb/VydZ2O9XsvsG5PKjpZ5teTQvjN\n9EE4aPp5jvblLEjdqTwu7ILqEuW8z8hGUQ+IEROlAkE3ECLeXWQZXg6BITfCgrVXbZp7uYZ//HiW\nr45l4m6v5o/XRXBbdACWfT1e3hH0Osg5Aak7IHWX4q3r60FlB8GTG0XdPVyEXgSCTiBEvLsUX4D/\njIYb/w1jl3Woy8nMUv72XSJHLpUQ6e3AMzcMZVLYAAsx1JbDpf2NnnpRinLe0V8JuYTNhJBpYNuP\n5hEEAhMgRLy7nPoSvloOD+4H7+Ed7ibLMltO57J6SxIZxdXMGuLJn68fQqiHvQmN7cWUXFI89NSd\ncGEP1F4GJPAb0+il+48TZQIEghYIEe8uW5+Cox/AU5lg2fmc8Bqtjg0HL/H6zhRqtDrumhjE72aG\n42zbh6sqdhddPWQfa/TSM4+ArAe1A4RMVTz1QTPAbZC5LRUIzI4Q8e7y3nUgWcCvt3brNgXltby2\n7Rz/dyQdRxsVv5sZztIJQagsRb411aVwca9B1HcoRcIAXIIbvfSQqaBxMquZAoE5ECLeHXRaWB0A\n0b+GuS/2yC2Tc8v4+3dJ7E8pJNTDjqfnD2F6hKfYxKIBWVbmIRq89It7oa4CJEvwjzaI+kxlVWkX\nvhkJBH0NIeLdIeckvD0Fbn0Phi/qsdvKsszO5Hxe+CGJCwWVTAl35+n5Q4nwduixMfoNOq0Sbknd\nCSk7IPsXQFa88pBpiqiHxoFriJkNFQhMgxDx7nD0A/juUXjkuElEQqvT80l8Gv/afp7yGi23jw/k\nD7MH424v8qrbpKoYLuxu9NTLspTzzoGKqIfGKaEXe08zGikQ9Bw9KuKSJHkBX8qyPKXJuW+BZ2RZ\nPn61vn1SxL/5LSR/D49fMGluc2lVHf/ecZ6PD6Vho7LkNzPCuHdSMNZWvWTnod6KLCurRi/sgYt7\n4OI+Q9YL4DkMQqcpwh4UK7ayE/RZekzEJUlyAT4FPGVZHmM4dycwTpblR9sboE+K+Jux4OgDS7+6\nJsOlFlTw4vdJ7EjOJ8DVhqfmDWFelLeIl3cUvU6p93JhtyLsGYehvkaJp/uNbRT1gPFiFamgz9CT\nIu4ISMA3sizHSZLkCiQCbwF7ZVnedbX+fU7E6yphtT9MfQym//maDr3/fCF//z6R5Nxyxge78swN\nQxnuLzIzOo22RhHyi3sUUc8+pqQyWtkohcxCpinC7j0CLMS3HkHvpMdj4pIk7TaI+N8AZ+AFYDWw\nSZblzS3argBWAAQGBo5NS0vrrP3mI+0gfDAP7vgcBs+55sPr9DL/dySDV386S3FVHQtH+/P43Ai8\nHNvZLk7QNtWlkHagMfxSkKyc1zgrFRlDpkHodCU/XXz7EfQSTCni3wF/kmU5WZKkecBsWZb/0Fa/\nPueJH1wLPz0Nf0oBew+zmVFWo+XNXam8v/8ilhaScTMKG7XwHLtNea6SwnhhjxKCKctUzjv6NXrp\nIdOUkJpAYCY6KuJdSbhNAUKBZCAa6ENudgfISlAyHswo4ACOGhVPzovkjvGBvLQ1iX9uP8dHhy7x\n68khLJ0QhJONWKbeZRy8YcRtyqMhP/3CbsVLP7cFTvxXaec+uDHzJXgy2Dib0WiBoHW64on7Au8C\nTkAVsFCW5fK2+vU5T/xfI5QFJbd9aG5LmpGQVszrO1PYdbYAe2sr7owJZPnkEDxFmKVn0esh71Sj\nl55+CLRVyupdn1GNXnrgBFDZmNtaQT9G5Il3hcpC+McgmP03mPSIua1plcTsMtbtSeW7k9lYWVhw\n61h/HpgaSrC7XfudBZ2nvk5ZdNQwSZp1VCm1a2mtZLuEToOgSeA7BlTiA1XQcwgR7wrnfoL/LoZl\nP0DwJHNbc1XSi6pYvy+Vz49mUq/TM2+4Dw9NG0SUn8hmMSm15ZB2qDH8kndaOW9praQzBk2EwFhF\n4EWOuqAbCBHvCrtWw96X4ckMsO4bpWMLymv54MBFPj6URnltPVPC3XkobhATQ91Envm1oLJI2bou\n7aDyyDkBsk4Jv3hFKV56g7CbeZ5F0LcQIt4VPlmkLOd++JC5Lek0ZTVaNsan897+ixRW1DIqwJmH\n4gYxe4gXFgNhh6HeQm2FEn5JP6SIeuZRqK9WrrmFQeBEZSVpUCw4B4mURkGbCBHvLLKsxMMj5sFN\nb5jbmi5To9Xx1bFM3t5zgfTiKgZ52PHgtEHcNMoPtZUof3vNqa9TVpOmHVSEPf0Q1BhKBDj4Grx0\ng7B7DAEL8TsSKAgR7ywll+DfI+GGfyolaPs49To9W07n8tbuVBJzyvBx0nDflFBuHxeAnbUo5Wo2\n9HooSGoMv6QfgvIc5ZrG2SDohvCL7yix49EARoh4Zzn9FXz5a3hgr7Jrez9BlmX2ni/kzV0pHL5Y\njLOtimWxwdwzMRgXuwG8y1BvQZYVB6Ih/JJ+qHFfUisbCBinCHrQRGUbO7XIQhooCBHvLD/+BY68\na9iOrX96PwlpJazbk8q2xDxsVJb8anwg900JwddZ5Dv3KiryGwU97aCSASPrwcJKcTACJyoTpoET\nxIbT/Rgh4p3l/XlK/u9928xtick5n1fOuj0X+Oa4UpP75tF+PDgtlDBPsTlFr6TmMmQcgXRDCCYr\nAXR1yjWPIY1xdf9ocAkRk6X9BCHinUFXDy8FwJh7YN5L5rbmmpFVWs27+y7w2c8Z1NTruG6oFw9O\nG8ToQBdzmya4GtoapTKjcbL0MNQZFk3buin56n7R4D9WeW4jfp99ESHinSH3NKybBAvfhRGLzW3N\nNae4so4NBy/x4cFLXK7WMjHUjYfiBjEl3F3kmvcF9DrIT1TSGbOOQmaCoVKj4X/bLcwg6tGKqHtF\ngZWYD+ntCBHvDAkfwrePwMpjSjnSAUplbT2f/pzOu/sukltWwzBfRx6KG8S8KB8sRa5536KmTNmX\ntEHUs45CRZ5yzdJaia37Nwh7tFL0TXxg9yqEiHeGzY9A4jfwxCXxhwzU1ev5+ngW6/akcqGgkmA3\nWx6YNoiFY/zE1nF9FVmGyxkGbz1BOeYcV3ZAArDzaBKCiQa/Mcqm1AKzIUS8M6ybrPwR37XJ3Jb0\nKvR6mZ8S83hrdwonMi/j6WDN8skh3D4uECfb/pnBM6DQaSHvTHNvvfCc4aKklOJtCMH4Ryv7l1qK\nNQbXCiHiHaWuStmObcofYMbT5ramVyLLModSi3hrTyr7zheitrJgzjBvFo31Z3KYuwi19CeqS5VJ\n08yjjTH2qiLlmspWKcfb4K37RysbaYhvrybBlJtC9C9yTyoFi/zGmtuSXoskScSGuRMb5k5idhmf\nH83g6+NZfHsiGx8nDQvH+LFobAAhohxu38fGGQbNUB7QuBipIQSTdRQOrwfdWuW6vXdzb913NFiL\nVNVrifDED70BP/4Z/ngOHLzMbU2fobZex86kfL5IyGT32Xz0MkQHubA42p/5I3yxF0v7+y/1dcrG\nGQ0hmMyjUJzaeN11kDJx6jNCOXqPBDs389nbRxHhlI7y5XJIj4c/nDG3JX2W/LIa/vdLFl8czSC1\noBIblSXzorxZFO3PhBA3UUVxIFBVrHjr2ceVCdPck1Ca3njd0b+FsI8AR18RirkKQsQ7yr9Hgfdw\nWPKxuS3p88iyzPGMUr5IyOTb49mU19bj72LDrWP8WTTWnwBXW3ObKLiWVBVD7imlxnruSeVYeB5j\n/rqte3NR9xmprDgVlRwBIeIdo6oYXg6BWX+FyY+a25p+RY1Wx49ncvkyIZP9KYXIMkwMdWNxtD9z\no7yxVYtwy4CktkLJiMk5AbknlGN+Mui1ynVrR8Wpairs7oMHZFaMEPGOcH47bLwV7vkOQqaY25p+\nS1ZpNf9LyOTLY5mkFVVhb23F/OE+LI72Z2yQi1gVOtCpr4X8pEZvPeeEsoq6YTMNKw14DWsUdZ+R\n4Dm03+9pKkS8I+x5GXa9CE9liBn1a4Asy/x8sZgvEzL5/lQOVXU6QtztWDTWn4Vj/PBxEtUUBQb0\nOiX00lTYc05CrWFDDQsr8IhsIuwjFA++H/0f96iIS5LkBXwpy/KUJueigH/Ksjz7an17tYj/d4mS\nPvWbw+a2ZMBRWVvPD6dy+CIhk58vFmMhweRwDxaN9ee6oV5oVGJlqKAFDemORmE3HCvzDQ0kcA01\nhGKilMVJnkP6bEmBHhNxSZJcgE8BT1mWxxjOScCPgFqW5bir9e+1Ii7L8Eo4hF8HN79pbmsGNGlF\nlXyVkMlXx7LIKq3GUWPFglG+LB4bwAh/JxFuEVyd8twm3rpB3C83yYxROyhi7jVUCcN4DlXCM728\nFntPirgjIAHfNAi2JEm/BryAOX1WxEvT4V/DYf6rMO4+c1sjQFnmfzC1iC8TMthyOpfaej2DvexZ\nNNafm0f74enQv2Oggh6k5rIyYZp/BvISlSqPeWegprSxjb23QdyHGcR9iBKiUfeOLKoej4lLkrRb\nluU4SZLcgC+AOcC21kRckqQVwAqAwMDAsWlpaZ2x/dpwZhN8sQxW7FZWmQl6FWU1Wr47kcMXCRn8\nkl6KpYVE3GAPFkf7MyPSS2z6LOg8sqx47UZhT1KeF5xtLATWEJJp6bW7hoLFtQ3xmVLE3wHelWX5\ncMO5q/XrtZ74T8/A4XXwVJaordzLSckv58uELP53LJP88lpcbFXcNMqP+SN8GBPoImq3CLqHXgfF\nFwzeemKjyBdfwJjTbqUBj4gmwm44OviYLN5uShE/B2QbTo8CXpdluc3KUb1WxD+Yr3z63r/D3JYI\nOki9Ts++84V8kZDB9sR86nR63OzUzBziyeyh3kwJdxcTooKeo64KCs82D8fkJ0FFbmMbjXPzcIyX\nYTK1B8r4mqwAlizLg5sMsvtqAt5r0euUgvmjl5rbEkEnsLK0YHqkJ9MjPSmr0bL7bAHbEvPYciqX\nz49mYqOyZEq4O7OHejFziBeuduIblqAbqG2VUGvLcGtlkSLqDY+8RDjxWeMWeQBOAYqYxzwAYbNM\namaHRby1sEl7oZReS8FZ0FaKyoV9GEeNigUjfVkw0pe6ej3xF4rYlpjHtsQ8fkrMw0KC6GBXrhvq\nxeyhXgS5iQqLgh7Czk1ZHNh0gaAsK8kSDXH2Bu+9tsLk5gzMxT6/fALf/AZ+exTcw81tjaAHkWWZ\nU1mXjYKenKt4RxFeDsw2CLpIWxT0BUQ98auRlQDWTkrJTEG/QpIkRvg7M8LfmT9eF0F6URU/Jeay\nLTGPN3en8PquFLwdNcwaqsTRJ4a6iUwXQZ9mYHrib08FGxe4+xtzWyK4hhRX1rEzOZ9tibnsPVdI\ntVaHg7UV0yI8mD3Ui+mRnjhqxLZzgt6B8MTbQlutzDJP+p25LRFcY1zt1Cwaq5TFrdHq2H++kG2J\neexIzuO7kzmoLCUmhLoZwy6ilougLzDwRDz3FOjrxaTmAEejsmTWUC9mDfVCp5f5Jb3EOCn67Ddn\nePabMwz3c1ImRod5EeHlIOLogl7JwBPxrGPK0XeMee0Q9BosLSSig12JDnblyXmRpBZU8JNhYvTV\nbed4dds5AlxtuG6oN7OHehEd5IKVpYijC3oHA1DEE8DBFxx9zG2JoBciSRJhng6EeTrwcFwY+WU1\nbE9S4ugfH0rjvf0XcbFVMSNSCblMHewuNrgQmBWzTGxqtVoyMzOpqalpo5cJKcsBSxXYuV/7sQW9\nFo1Gg7+/PypV2xObFbX17D2nLDDakZRHWU09aisLxga6EDvIjdgwd0b4O6ESXrqgB+jVm0JcvHgR\nBwcH3Nzcrm2cUV+vxMQdfMDB+9qNK+jVyLJMUVER5eXlhISEdKiPVqfnyMVidibncyC1iKScMgDs\nra0YH+KqiPogdyK9HcRG0YIu0auzU2pqaggODr72E0V1VcpRLVbvCRqRJAk3NzcKCgo63EdlaUFs\nmDuxYco3uuLKOg6lFnEwtZBDqUXsTFY2KnC1UzMx1I2Jg9yYFOZOsJutmCAV9ChmC+aZ5Q9ZaxBx\nlUgdEzSnu3+PrnZq5o/wYf4IZa4lu7SaQ6lFHEgt5GBKEd+fygHAx0lD7CB3Q/jFTaQxCrrNwAre\n1VWBlTUVVTXccsstTJ48mXvuuYf6+vqrdnv00Uc7dPuOtmuNuLi4q16PiYnh/PnzAGzevJl7770X\ngGeeeYbY2FhuueUWKipar9OwatUqdu/e3WXbrsaGDRvYsGGDSe7dl/F1tuHWsf68dtsoDj01g51/\nnMbfb45idKAzO5Pz+OMXJ5i4eiczXtnNXzad4odTORRX1pnbbEEfZOBMq8uyUvTK2pG1a9cSHh7O\npk2buP322/n888+544472uz6r3/9q0NDdLRdV5g7dy7btm0jPDycHTt2MGfOHA4ePMi+ffs4cOAA\n69atY/369fzhD38wmQ2CriFJEqEe9oR62LN0QhB6vUxybjkHUws5mFrE179ksfGwsp3YUB9HYg2h\nl3EhrthbD5x/UUHXGDh/ITqtMrGpsuXw4cPcd5+yJdvkyZM5cuQId9xxB3FxcYwbN46TJ0/y448/\nGrvGxcUZPdnq6moWLlxIcXExgwYNIioqij//+c9XtFu1ahVarZZ9+/ZRVlbG1q1bsbe3Z9GiRVRW\nVhIWFsYHH3zQYfPnzJnDK6+8wsMPP8yuXbt49tln+c9//sP111+PJEnMmTOHM2fOtNn/tdde47nn\nnsPHx4eNGzdiaWnJypUrOX78OM7Oznz00Ud8841ShmDZsmXs3r2b3bt3ExwczIkTJzhx4gS5ubl8\n/vnnREREcNttt1FaWopKpbrqB6DgSiwsJIb6OjLU15H7poSi1ek5mXmZgymKqH8Un8a7+y9iZSEx\nMsDZOEk6OtBZ1EsXXIHZRfyv354hMbusR+851NeR524c1vxkQzxcbUt5eTl2dsrkpq2tLWVlyvjx\n8fE88sgj/OMf/2jz3snJyfj7+7N582YmTZrEf//73zbbpqSksHfvXp5//nl27tzJuHHjWLlyJbNm\nzWLu3Lnk5eXh5eXVofcUExPD8ePHyczMxNbWFjc3N/Ly8oiOViavQ0NDCQ0N5YEHHuDs2bPGfjNm\nzAAgOjqaZ599lgceeIBvv/0WKysrampq2LdvHxs2bGDNmjVERka2OvaRI0fYtWsXn376KZs3b2bQ\noEEEBQWxadMm7r///g7ZL2gblaUFY4NcGBvkwsqZ4dRodSSklXAwtZADKUW8sSuFtTtTsLayYFyw\nq3GSNMrXUSw6EphfxK8Z2kpAApUNjo6OxvhxZWUljo6OAERFRbFw4cKr3sbPz4+EhASmTp3K7353\n9ford999NwCBgYHU1dWhUql49913+eCDDyguLqa6urrD5ltaWjJ27FjWrFnDddddB9Dsffz888/s\n2bOHt99++4q+q1atIiYmBoAxY8aQmpqKTqcznpswYQKbNm1qJuJNbfvVr36FSqUiMDCQCxcucPHi\nRUaOHAlg/BAR9BwalSWTwtyZFObOY3OU/UZ/vlDMQUP2yz9+PMs/fjyLg7UVMaGuxA5yZ0KoGxHe\nDmKrugGI2UX8Co/ZVNRVKVkpkgUxMTHs3r2bG2+8kX379nHLLbcAYG9v3+5ttm7dyjPPPGPsczUa\nvP0G3nvvPRYtWsRtt93GtGnTOv0W5s6dy4MPPsiePXsAmDRpEuvXr+fRRx9lz5492Ni0nemQkJDA\nnDlzOHnyJDNmzECj0bBp0ybuu+8+4uPjGTZsGGq1mqysLAC2bNmCq6trq+8jMDCQY8eU8gW//PIL\nEyZM6PR7EXQcR43KWOcFoLCilvgLRRxIKeJQaiHbk5R0RntrK0YGODE6wIUxQc6MDnDBRexu1O8x\nu4hfE2RZCafYKKL029/+lrvuuovY2FjCw8NZvHhxh281evRo5s2bx9q1a/H09OTpp58mKiqqQ31n\nz57Nww8/zLp16wDIysoiODi4w2PPmTMHe3t7owe9YMECtm/fTmxsLO7u7nz66adt9t23bx/Tpk3D\nw8ODm266CSsrK7Zu3crkyZNxcXHho48+ora2lttuu41Lly5d1Y5bb72Vzz77jLi4OCwsLISIX2Pc\n7a25YYQvN4zwBSCzpIojl4o5llbKLxklvLUnFZ1eWcQX6m7HqEBnxgS6MCbQRXjr/RCzrNhMSkpi\nyJAhJh23GdoaKEgC50CwdevWrd555x0+/fRTVCoVKpWKP/3pT+2mBwr6Btf879JEVNXVczLzMsfS\nS/glvZRf0ksorFDSF+3UlowMcGa0QdhHB7qIvUh7Kb16xeY1R1upHFXdX6l5//33i8k8Qa/GVm3F\nhFA3JoQqDossy2QUVxtEvYRj6aWs23PB6K0Hu9kqgh7kwphAZyK8HMSEaR+iQyIuSZIX8KUsy1Mk\nSQoEPgL0QArwgGxqd7671FWBZAFW1ua2RCC45kiSRKCbLYFuttw82g+A6jodJzNLOWbw1PeeL+R/\nvyjzIbZqS0b4OxlDMKMDnXGzF/87vZV2RVySJBfgQ6DBjX0AeEiW5SRJkrYAw4GTpjOxB9BWgcoW\nRM0KgQAAG7UlMaFuxDTx1jNLqo0hmGPpJazfe4F6g7ceZPDWxwQ6MzrQhUhv4a33FjriieuAJcA3\nALIs/6XJNTeg0AR29RyyXtmSzd7D3JYIBL0WSZIIcLUlwNWWm0Y1euunsi4bQjAl7E8pZJPBW7dR\nGbz1oEZv3V1462ahXRGXZbkMriwQJEnSEuCMLMvZLftIkrQCWAFKOppZ0VYDco/EwwWCgYSN2pLx\nIa6MD1Gyulp667+kl/BOE2890NWWkQHORPk6MszXiWG+jiLF8RrQpYlNSZJCgT8Bs1q7LsvyemA9\nKNkpXbauJ2goP6uyNZ6KiYnhk08+ITw8nM2bN7Np06ZOLYEHpdhVV2qlHD9+HIBRo0a12aYt+555\n5hl27NiBl5cXH3/8cat57atWrSIuLs4kGTMNha6WLVvW4/cW9H5a89ZrtE289bRSjqWV8O2JRr/O\n10nDUF8novwahd3HSSPK8fYgnRZxQ4z8U+DXsixf7nmTehhtFViolN18DLRWTKqzdLXYVUdEXBS7\nEvQVNCpLxgW7Mi7Y1XiupLK9Ga1NAAAgAElEQVSOM9llnMm+bDzuSM6jIf3BxVZlFPRhfsoxxM1O\nbJ7RRbriiT8JBAJrDZ+mz8myvKdHrepJ6iqvmNRsrZhURUVFq8WpOlIUq7W+rRXA+ve//82mTZsA\n+Pjjj9mxY0erJotiV4K+jIudmsnh7kwOb9wCsbK2nuTcMkXUs8o4k3OZDw5cok6nB5SMmCE+jgzz\ndSTK14mhvo4M9nJAbSUmT9ujwyIuy3Kc4fgE8ESPWbDlSWXLtJ7EezjMe0mpWqirBVvXZpdbKyZ1\n/vz5VotTdaQoVk5OzhV94coCWKtXryYiIgK4ekhCFLsS9DfsrK0YG+TK2KDG/8W6ej3n88s5k11G\nosFj/yohk48OpQGgspQI93RQhN3gsQ/xccROlOdtRv/+aWgNRZyaxMOh9WJSbRWn6khRrLb6tiyA\n1VFEsSvBQEBtZWEIqzgZz+n1MpeKKg1hGEXYdybn80VCJqB8oQ5xs2NoE2Ef5us0oFedml/E571k\nunvXGVZqqm2vuNSymFRbxak6UhSrrb4tC0cB2NjYUFRUBCiz/W1N8IhiV4KBiIVF4wYaN45UasPI\nskxuWY0ShjEI+y/ppXx3MsfYz8dJYxT0Bo/dz9lmQMTZzS/ipkRbBZbWYHHl22xZTKo7xala63u1\ntrfddhsbN25k9erVTJ06tdV2otiVQKAgSRI+Tjb4ONkYKzmCMoGamNN0ArWMHcn5xglUW7UlYZ72\nDPZyYLCXPeFeDgz2csC3n2XH9O8CWLmnwdoeXIJNP5agz9NfCmANZKrq6knKKedcXtNHBQXltcY2\n9tZWhHvZM9jTQTl6ORDh7YCng3WvEndRAEtXB3rtFfFwgUDQf7FVWxl3SWpKaVUd5/IqOJtXznmD\nuG9PyuP/jmYY2zhqrBjs5WDw2O2JMDx3t1f3KnFvSf8V8YZFPmqxUlMgGOg426qbrT5toLCilnN5\n5ZzPqzAefziVw6c/a41tXGxVhHs5ENEiLNNbJlP7r4hrqwAJrNqeABQIBAMbd3tr3O2tiR3UmNMu\nyzIF5bWcMwh7w+PrX7Ior61v0ldtiLc3hmUGezrgZKtqbSiT0X9FvK4SVBqwEIsFBAJBx5EkCU9H\nDZ6OmmYLlhqyZM7lVXAu1yDu+RV8cTSDyjqdsZ2Xo7Ui7J4OLBjly6gAZ5Pa2z9FXJaVHHEbl/bb\nCgQCQQdomiUzbXBjVVS9XiartJrz+eXNvPf//pzGUF9Hk4t4/3RT62tB1rWaHw7KasnRo0czceJE\nFi9ejFarbbVdyz7tpeN1hZ4uVBUXF8fEiROZNGkSd999Nzqdrv1O7XDp0qUrVph+/fXXlJaWdvpe\nISEhxMXFERsby5o1a7ptW1PENnkCc2BhoRQGmxHpxYPTBvHabaP4buUUzvx1LgsMue4mHd/kI5gD\n7ZWVC1uydu1aDh06hL29Pdu3b79Ghl0bvvjiCw4cOIBarTbZe+uqiFtaWrJ7924OHDjAxx9/TGVl\npQmsEwjMj6WFdE1qv/RTEa80bMemuWozWZapqKhArVaTnZ3N5MmTmTJlCn/5i7LvxcWLF4mNjWXW\nrFkkJiYCkJeXx7x584iNjWX16tWAUstk8uTJLFmyhFGjRvHDDz9w6tQpxo8fz6RJk3jzzTcBSE1N\nZebMmYwfP54nnmgsP/OPf/yD2NhYFixYgCzLnDt3junTpxMTE8NHH30EwO7du/n973/PLbfcYrSv\nvfdWWlqKjY1Nq/eLi4u7YtzWbG6NefPmsXXrVm6//Xb++Mc/AjB9+nTjIqeZM2dedcETQE1NDVqt\nFktLSy5evMjMmTOJjY011qhp6lU3fAtqzeaEhATGjh3L/Pnzyc3NbffnIhD0N8weE1/z8xqSi5N7\n9J6RNl48MWz5VbdjW7lyJcXFxdx4443MmDGDo0eP8tJLLzFq1ChiY2N54YUXePnll3n88ce54YYb\nGD58OACrV69myZIlLFu2jJiYGFasWIEsy3z44YcsX76cV199lf3796NWq3n//ffx8fFh7ty5PPzw\nwzz22GP8/e9/Z+LEiTz22GPGWih2dnYcPHiQGTNmkJOTw+OPP85zzz1HbGwso0eP5q677gLg888/\nZ8+ePYSFhV31/S9evBhLS0umT5/O1KlTufnmm1u9X8tx8/LyrrC5NbZs2cKyZctYtWqVcVXr0qVL\n+eyzz7jzzjtRq9X4+fm12len0xEXF0dycjJLly5Fo9Hw2GOP8fzzzxMbG8vs2bO54YYb2nxvLW1+\n+umnee+994yFugSCgYbZRdwk6OvbXeSzdu1a9u/fj7W1skrLysqKv/71r9jb21NeXg5gLP5kZWVl\nrP999uxZDh06xIYNG6isrCQ7O5ugoCAsLS0JDg7G0tISWZaRZZnHH38cd3d36uuVtKTk5GTGjRsH\nwJo1a7AwZM7cc889QGOhrLNnz/Lcc88hSRI6nc4YtvjVr37VroCDEk7x9/c3vm7rfi3Hbc3mjrJ4\n8WLmz5+PWq1m6dKlbbZrCKdUV1czbdo09u3bR1JSEjExMUiSRHR0NMnJzT/UmxboamnzpUuXGDFi\nBBYWFsYKkQLBQMLsIv7E+J6ragsoqYWF59qc1GzKAw88wJQpU3jkkUd47bXXeOqppxg5ciQjRowA\nFKE4c+YMgYGBnDqllMuNiIjgpptuYvr06XzyySfGwlEtef755/nss8+wtLQ0ViKMjIzk6NGjTJgw\ngTlz5hhDFi2LTkVERPDPf/6TkJAQXn/9ddRqZVFBR4pxtUZb92s5bms2t4WNjQ1VVVXG146OjgQE\nBPDOO+8QHx/frk02NjbY2tpSXl7O0KFDiY+PZ9KkSRw5coRly5ahVqspKCjA1taWffv2Gfu1tDkg\nIIDExERCQkI4f/58u+MKBP0Ns4t4j2Oc1Gx/paaLiwszZszgq6++4oYbbuDBBx/Ew8MDW1tbsrKy\nePzxx1m6dCmvvfaaUfiefPJJli9fztNPP01ISAi33357q/deuHAh119/Pf7+/mi1Wmpqanj55Ze5\n//770Wq1zJw5k/Dw8Fb7vvTSS9x///1UVFQwadKkVqshdoaO3q81mzWa1ucV7r77buN7aaiquGjR\nIqytrbG1bfsDtCGcUl5eTkREBLNnz2bIkCEsX76c6upqFi5cSGRkJHfffTf33HMPgYGBREVFtXm/\n559/nnvuuQcPD482P1AFgv5M/yuAVZIGtWXgFXXVmLigZ/nkk0/45z//yfvvv2+sP97XEAWwBL2J\ngVsAS1t1xXZsAtOzdOnSq8bCBQKBaehfKYZ6HdTXiKJXAoFgwNC/RLwDi3wEAoGgP9G/RLxOiLhA\nIBhYdEjEJUnykiRpn+G5SpKkbyVJOiBJ0q9Na14n0VaCpRos+1+oXyAQCFqjXRGXJMkF+BBoCDSv\nBBJkWZ4ELJIkycGE9nWOuqoOpRauWrWKIUOGMHXqVGbOnEl2dvY1MO5KG3bv3m22MWJiYox51Zs3\nb+bee++9JuN2lIYiZXFxcSxZsqTbhbw2bNjAhg0bunUPgaA30hFPXAcsAcoMr+OAzw3P9wLtpsBc\nE3RaZTu2DizyAfjLX/7C3r17uffee1m7dq2Jjet9zJ07l23btgGwY8cO5syZY2aLrmTt2rXs3r0b\nFxcXfvrpJ3ObIxD0StqNO8iyXAY03WPODmioblQMeLXsI0nSCmAFKKserwldjIeXlJRgY2NDRUUF\nixYtorKykrCwMD744ANAKcQ0btw4Tp48yY8//tjqudraWpYtW0Z2djb+/v588MEHvPjii2i1Wvbt\n20dZWRlbt27F2tqaxYsXo9PpkGWZuLi4Vvs2LCxqSmv2rVq1qkNjtMacOXN45ZVXePjhh9m1axfP\nPvtsh8fw9vZu9efYctyqqiruvvtu8vPzGT58OG+88UaH79eUwsJC4wKljvzs6+vrrxi3gTNnzvDb\n3/6WzZs34+DQe75ECgRdpSsTmxVAw55n9q3dQ5bl9bIsR8uyHO3h4dHycjNyX3yRtLvu7v5j+QOk\n/f5vpN33MLkvvtjum3jhhReYOnUq8fHx/O53vyMnJ4eVK1eyfft2Ll26RF5eHgDx8fFMnDjRKOCt\nnXvnnXeIiopiz549hIeH8/777wOQkpLC3r17WbhwITt37mT9+vXccMMN7Nq1C5VKddW+LWnLvo6M\n0RoxMTEcP36czMxMbG1tcXNz6/AYrdHauOvXrycqKoq9e/eSk5PDyZMnO3w/UIqURUZGkp2dzcSJ\nEzv8s29r3JycHO68804+/fRTIeCCfkNXRDwBmGx4PhK41GPWdAdZp5Sf7SAN4ZSNGzfi5OSESqXi\n3Xff5c4776S4uNhYdCkqKoqFCxc269vyXGJiIjExMQBMmDCBpKQkQFmaDo3FmhoKagFER0dftW9L\n2rKvI2O0hqWlJWPHjmXNmjXGOikdHaM1Whv37NmzbNq0ibi4OC5cuGAsT9uR+4ESTklMTGTcuHG8\n9NJLQMd+9m2N+/rrr+Pv709aWlqbYwoEfY2upHF8CPwgSdIUYChwuDsGeP/5z93priDLkHsKbJzB\nuWvhm/fee49FixZx2223MW3aNOP51opOtTw3bNgw4uPjmTVrFvHx8QwbNozs7OwrapQ0FNSaPn06\nx48fZ86cOa327Yx9HRmjLebOncuDDz7Inj17OjVGa7Q2bkREBOPHj+fee+/lu+++IzAwkMOHD3eq\nFoyFhQUuLi7GypId+dlXVVVdMW5eXh7PPPMM8+fPZ/ny5SLGLug3dNh1lWU5znBMA2YDB4BZsix3\nf/+v7qIzbMfWjfzw2bNns3r1ambMmAHQ7qYGTbnvvvs4c+YMU6dO5fz581dsZdbAihUr+Oqrr4iL\ni6OsrKxTfTtqX2tjtMWcOXOwt7c3erLd+Rm0Nu7999/Pli1bmDp1KuvWrSMgIKDD9wMlnDJp0iS2\nbdvGb37zm1bbtPbza2tcjUZDQEAAkZGRbN68uVO2CAS9lf5RAKuqGErTwCMSVDbttxcIWkEUwBL0\nJjpaAKt/rNjUVnVoOzaBQCDob/QPEa+rUjxwUblQIBAMMPq+iMt6Q/lZUblQIBAMPPq+iGtrALnD\nKzUFAoGgP9EPRFxULhQIBAOXfiDilWBhpVQv7CDLli1j1KhRREdH884775jQOIFAIDAtfb9ma13X\ntmN7/fXXGTJkCCNHjiQmJsa4w71AIBD0Jfq2J27cjq1roRQ3Nzfmz5/Ppk2bmDdvHrGxsaxevRpQ\nFqocP34cgAceeIAjR45w6dIl7rzzTu69915j6dbi4mJuvPFGpkyZwqOPPgrAPffcQ3x8PKB4/fHx\n8eTl5V0xhkAgEHQXs3vi+z4/R2FGRdc6yzrQVoOVFixyjKfdA+yZctvgDt3Czc2Nv/3tb7z77rss\nW7aMmJgYVqxYwaJFi9iyZQujRo0iOTmZcePGcenSJb799lt++uknJkyYAMCLL77I7bffbhT3rVu3\ncvfdd7Nx40bGjBlDUlISEyZM4NFHH2XJkiXNxnBzc+va+xYIBAIDfd8TB7Cw7PItiouL0el0vPXW\nW8TFxVFZWUl2djYzZ87k8OHDJCUlNSskdd111xkFHJoXYIqJiSEpKYnp06dz6NAhvv/+exYsWAAo\nxaBajiEQCATdxeyeeEc95lYpvqhkp3i1XjSqPUpLS9myZQtz5szhiSeeYPr06XzyySe4urpiZWWF\nh4cHn376KYsWLTL2aasAU1hYGPHx8dxxxx1YWFgwe/Zs/vSnPxlLrUZERHDTTTc1G0MgEAi6S9/2\nxLVVXU4tXLlyJXPnzmXNmjVs2LCBV155hUmTJrF161a8vJR9Lm655RY+++yzZp53S5566ik+++wz\nJk+ejLOzs7Gs6+LFiwkMDCQoKAiAJ598stUxBAKBoDv03QJYOi3knQZHP7D37N69epgdO3bw+OOP\n88ILLzB37tweuadep0eSJCQLUVrAVIgCWILeREcLYJk9nNJlevEin5kzZ5KQkNBj95NlmdL8aiws\nJZzcbYSQCwQCI303nGLcU7P/l56VJAkbexV11fVcLqhG1pv225NAIOg7mE3Eux3G0VaClU23MlP6\nEjYOahxcNdTVCCE3BaYOKwoEpsIsIq7RaCgqKur6P44sK574ACt6JYTcNMiyTFFRERqNqEcv6HuY\nJSbu7+9PZmYmBQUFXbuBvh7KcsC2FnIqe9a4PoC2VkdNhRbLSxbYOKiQRB31bqPRaPD39ze3GQJB\npzGLiKtUKkJCQrp+g1Nfwo/L4YF94GPabIJLly9xvOA4AQ4BBDkG4aZx6xWimXggm10bkgkY4sr1\nDw7HSj0wwkoCgaA5fTM7JeuYEg/3NK2AV2mreGj7Q2RWZBrP2VrZEugYSKBDIEGOQUZxD3QMvKYC\nP3SSL5IEOz9O5oe3TnL9QyOEkAsEA5BOi7gkSS7ARsATSJBl+YEet6o9shLAZyRYqkw6zOvHXyez\nIpNXpr2CncqOtLI00svSSS9PJ7k4mR3pO9DJOmN7O5UdgQ6BRpEPdGwUelMI/JBYX0Bi58dJfP/m\nSa5/eAQqIeQCwYCiK574XcBGWZY3SpL0X0mSomVZPtpur55CVw85JyD61yYd5nj+cT5J/IQlEUuY\nEzwHgMl+k5u10eq15FTkKOJenk56WTpp5WkkFiWyPW37VQU+yDHIKPCuGtcuC/yQWB8kCXZ8lMQP\nQsgFggFHV0S8CIiSJMkZCAAyetakdihIgvpq8BtjsiFqdbU8e/BZvO28+f3Y37fZTmWhUkTZMfCK\na1q9luyKbNLK0sgozzB68a0JvL3K/grvveG5i7VLuwIfOdEHJNjxYRLfv3GS+b8RQi4QDBS6IuL7\ngfnAI0ASUNyygSRJK4AVAIGBVwpct8gyrIQ0oYi/feJtLl6+yLpZ67Dr4gbMKguV0dtuiVanJasi\nq9F7Nwj96cLT/JT2E3pZb2zroHIgwDGAIIcgQp1DGeI6hAjXCLxsvZqJe+QEHyRguxBygWBA0RUR\nfw54UJblMkmS/gDcC6xv2kCW5fUN56Kjo3s2mTkrAWxcwKUb2S1XIbEokfdPv89Ng25ikt8kk4yh\nslQR7BRMsFPwFdeaCnzTGPzJwpNsubTF2M7F2oUI1wijqA9xHULY+CCQJHZsSOT7N04w/+GRqKyF\nkAsE/ZmuiLgLMFySpHggBtjesya1Q9Yx8Bvb6e3YOoJWr+XZA8/ionHhsXGP9fj9O8LVBL5SW8m5\nknMkFycbH58kfYJWrwVAY6lhsMtghk+ejLx/MP/3r4Ms+O1oHO3sr7iXQCDoH3RFxFcDHwBBwCHg\n0x616GrUVUJ+IkTON8nt3z/1PmdLzvLv6f/GydrJJGN0BzuVHaM9RzPac7TxnFav5eLli82EfbPF\nx3gNGsyMlKWs+etGksb/RJjnoGZeu4vGxYzvRCAQ9BSdFnFZln8GurYLQ3fJOQmyHnx7Ph6eUpLC\nupPrmBs8lxmBM3r8/qZCZaFisMtgBrsMZsEgZRchWZbJrszm8L5kLDaH4XzChR3DNrDlYmM4xtPW\ns5moR7hG4G/v3ysWMgkEgo7Ttxb7mGhSU6fX8ezBZ3FQOfBUzFM9em9zIEkSfvZ+LJznxzn3XLa/\nn8iKzL8x5f5gUqvOc7b4LEnFSZwtPsv+rP3GTBl7lf0VcfZQp1BUJs7HFwgEXafvibhTYI9vAvFJ\n0iecKjzFmilrcNX0r23TBo/zRpIktr2fyL53LnHDb6OZ4NO4U1FNfQ0ppSnNwjFfnf+K6vpqQPH0\nw5zDiHCNINI1kkjXSMJdwnFUO5rrLQkEgib0WhGX6+vB0rL51/ushB73wtPK0lj7y1riAuKYFzKv\nR+/dWwiPVraC2/Z+It+9foIbfjsStUb51WusNES5RxHlHmVsr9PrjKtSGx57M/fydcrXxjaeNp6E\nuYQxyHkQYc5hhDkrz7uakikQCLpGrxXxy19/TeGbb+EwezYOs2dhMzgQqTQNxt3XY2PoZT3PHXwO\ntYWaZyY806/jweHRSl75T++d4bu1J7hhZaOQt8TSwpIQpxBCnEKMH2yyLFNQXUBycTIppSmklqZy\nvuQ8X5z9ghpdjbGvj51PM1EPcwkj1CkUG6v+v3mHQGAOeq2IW/n4YD14MCX//S/FH36IpbMDDq5O\nOESpsaurQ1Kruz3G52c/JyEvgedjn8fTtnft02kKwsYq77EjQt4SSZLwtPXE09aTqf5Tjed1eh1Z\nFVlGYU8pTSGlNIX4nHhj6qOEEqMPcw5r5r2HOIVgbWnd829UIBhAmGWj5M6gq6ikct9eyjf+h4rj\nF9DXW2Bhb4/9tGk4zJ6N/ZTJWNh1/it8dkU2t3xzCyM9RvL27Lf7tRfekpSEfH567wxewY7cuHIk\napue/yyv19eTUZ5hFPXU0lRSSlJIK0ujXq4HwEKyINAh8IqQTLBjsJhMFQx4OrpRcq8XcSMbF6Mv\nyqByxEuUb99OxY6d6EpKkKytsZs0CYdZs7CfHoeVS/v5z7Is8+D2B/kl/xc23bQJP3u/7tvXx0g9\nls9P757BM9iBG1eOMomQt4ZWpyWtLK25uJemkF6ebiw3YCVZEeQY1CjuBu890CEQK4te++VRIOhR\n+peIyzL8YxBEzIOb3lBO1ddTdewY5du2U759O/U5OWBpie24cTjMnoXDrFmovLxavd3XKV/zzIFn\n+HPMn/lV5K+6Z1sfJvWXfH5659oLeWvU6mq5dPkS50vPN4ZlSlLIqshCRvkbVVmoCHEKMYp7kGMQ\nwY7BBDgEYKsaWFv1Cfo//UvES9Lg3yNg/mswbvkVl2VZpub0Gcq3b6d82zbqLlwAQDNyBA6zZuE4\nezbq4GAACqoKuOmbmwh3DueDuR9gIZltr+hewYVfCvjxndN4BDlw4yOjsDajkLdGlbaKi2UXSSkx\nTKYaRD6nMqdZO08bz8YKkIZjkEMQAY4BIu4u6JP0LxE//T/48l5YsQd8R7XbvDY11eih15w+DYB1\neBj2s2azzvkYmy1O8eWCr1qtTzIQ6e1C3hqV2kpj/faGSpANBcNKakuM7SQkvO28FWF3CDJWlgx0\nDMTf3l/E3gW9lv4l4j/+BX5+B/6c1endfLTZ2ZRv30H5tm1UJhxF0svUeDrhO+9mHK6bjc2oUUiW\notLfheOKkLsHOLDgkZFY2/ZdcSurKzMKe4PQp11OI608jfK6cmM7S8kSHzufZsLe4MH72PuI+LvA\nrPQvEX9/nrLD/X3bunyL4ppilm5cwLRLttyeH0L1wUPIWi2Wbm44zJyJw+xZ2MXE9EjqYl/l4okC\ntq4/jbu/PQt+N6pPC3lryLJMaW2pcSempt57WlkaVfVVxrZWFlb42/s3E/aG59523gM+DCcwPf1H\nxHX18FIAjLkH5r3U5ds8vvdxtqVt4/MbPifcJRxdRQWVe/dStm0blXv2oq+qwsLBoXnqou3Amyy7\neLKQrW+f6rdC3hayLFNUU9RM1NPKFO89oyyj2YImtYW62QbZfvZ++Nr74mfvh4+dj5hkFfQI/UfE\n887AW7Gw8B0YcVuXbrErfReP7HqEh0c9zEMjH7riur62lsqDBxtTF0tLldTFyZOV1MW4aR1KXewv\nNBXyGx8ZhcZuYAh5W+hlPflV+c1i8JfKLpFelk5GeYZxUVMDLtYu+Nr7GoXdeLRTzgmRF3SE/iPi\nxz6CzSth5TFwG9Tp7mV1Zdz89c24aFz4bP5n7U5kyfX1VCUco3zbNiV1MTcXLCzQDI/CftJk7CZP\nxmbEcCSr/h0vvXSykC3rT+Hmq3jkA13I20Iv6ymqLiKrIovsimyyK7Mbnxsedfq6Zn1crF3wsfdp\nJuwNYu9r7yvqzwiA/iTi3/4OzmyCJ9K6tJvPsweeZXPqZjbO38gwt86VQVdSF09TsXsPlfv3U33q\nFOj1WDg4YDdxInaTJ2E/eTIqX99O29UXuHSqkC1vCyHvDnpZT3FNsVHYmwp8w/OWIu9s7awIukHg\nW3r0QuQHBv1HxNdNATt3uGtTp7sezDrIA9sfYHnUch4d+2jXbTCgKy2lMj6eiv37qdx/QPHSAXVo\nqFHQbceNw8Km/xR7EkJuWpqKfE5FTqPYVzaKfa2utlkfJ2snfO0MMXiDR+9j54OXnRdetl64alzF\nxGs/oH+IuLYaXvSDyb+Hmc90qmultpKF3yxEbanmywVf9viCD1mWqUtNNQp61ZEjyLW1SGo1ttFj\nsTOEXqwHh/f5uiwNQu7qY8dNj44WQn4NaZhwbeq951TmNPPom066gpJZ42mjFCtrEPaG59623nja\neuJh64HKQvweezP9Q8TTD8P718Htn0Lk9Z3q+kL8C/zf2f/jo3kfMcqz/QVC3UVfU0PV0QQq9++n\n8sB+as+nAGDl6YndpEnYTZ6EXWxsn50gTTtdxJZ1p3DxseX6h0bg4Koxt0kCFJEvrikmpzKHvKo8\n8irzyK/KJ6+q8ZhXmXeF0EtIuNm4KeJuqwh9g+A3iL6nraeYhDUj/UPED70JPz4FfzwLDt4d7paQ\nl8CyrctYOmQpT4x/omtjdxNtbi6VBw4onvrBQ+gvXwZJQhMVZQy92Iwc2acmSNPOFLHlrVPodXr8\nh7gSOcGbkFEeqNRisVRvRpZlyurKrhB546NSOTZdCNWAo9rxCi++QfAbnjuqHfv8t83eiMlFXJKk\nN4Etsix/e7V23RLxL5dDejz84UyHu1TXV7No8yJ0so7/Lfhfr/AkZJ1OmSA1hF6qT5xQJkjt7bGb\nOMEYelH79/5qipcLqkk+lENyfA4VxbWoNZaEjfUkYqIPPoOcxD9zH6ZKW0V+VX5zka9sFPv8qnyK\nqouMBckasLGyMQq6h60H7hp3PGw9cLNxw93GHXeNO+427jhZi7+PzmBSEZckaQrwe1mWF7bXtlsi\n/p/R4BUFSz7ucJdXj77KhjMbeOe6d5rtJdmb0JWVUXkonsr9+6k4sJ/6bKWYkzo4GLvJk5XQy/jx\nvXqxkayXyTpfytlDOZ/GWE8AABJpSURBVKT8UkB9rQ5HDxsiJ3gTEeONo3v/mdwVNKLVaSmoLiC/\nKp/cqlzyK5sLfkF1AYXVhVdMxoISq28q6u627q2+dtO4obES4TqTibgkSSrgFPADsEeW5W+u1r7L\nIl5VDC+HwKxVysRmBzhVcIqlW5ZyS9gtrIpd1fkxzYAsy9RdvKgI+v79VP18BLmmBkmlwmbsWOwn\nT1ImSCMieq0XU1dTz4VfCkiOzyHrbCkAfoOdiZjgw6AxHh3ePUjQP5BlmQptBYXVhRRWF1JUXWQU\n95avS2pKrvDsARxUDrjZuBk9+wavvuVrF41Lv83EMaWILwfmAw8DK4FcWZbXtmizAlgBEBgYODYt\nLa1TYwCQsh0+uRXu+RZCprbbvE5Xx5LvllBWV8bXN32Ng9qh82P2AvS1tVQnJFCx/wCV+/dTe+4c\nAJYe7tiNG4dmWBSaqCg0w4ZiaW9vZmuvpKywmrOHc0mOz6WsoBora0sGjfYgcoI3foNdkCx65weR\nwDzU6+spqSkxinpRdZFR7AuqC5q9blrbpgFLyRI3jVtj6MbwcLNxw8XaBVcbV1ysXXCzccPZ2rlP\nFTUzpYi/Dnwny/JWSZKGAC9cLazSZU98z8uw60V4Mh00ju02f+P4G6w7sY43Zr7RbA/Ivo42L5/K\nA4qgVx8/jjY7W7kgSahDQtBEDcMmyiDsQ4b0mhx1WZbJTb1McnwuKUfzqKvRYe9qTeQEHyJivHH2\n6r2hIkHvpEpbZRT01kTe6OnXFBl3iWqJk7WTIu4aV+PDRdP6a2drZywtzDdpb0oRfxSok2X5TUmS\n7gLGyLLcZryjyyJeWw75yRAwrt2mZ4vPcvt3tzM3ZC6rp6zu/Fh9iPriYmrOnKHm9GmqTyvH+rw8\n5aKFBdZhYYqgG8TdOiICC2vzbopQX6fj4olCkg/lkJFUjCyDd6gTkRO9CRvrOWCKbAmuDTq9jrK6\nMoprio2PkpqSK543HEtrS1sN6UhIOFk7XVXom752Ujv1qOibUsQdgPcBL0AFLJJlOaut9j22x2Yb\n1OvrueP7O8iryuObm77BWeNssrF6K9q8/EZhP3OamlOn0RUXKxdVKjTh4c2FPTwcSWUe4awsrTWG\nW0pyKrFUWRA60p2IiT4EDHHFQoRbBNcYnV5HaW1po9DXXin0TT8ALtdeblX0LSQLnK2djaLuYu3C\nrYNvJdY3tkt2dVTEOx0gkmW5HFjcJatMwIYzG0gqTuLVaa8OSAEHUHl5ovLyxGHGdEAJZdTn5FB9\n+jQ1Bm+97McfKf38cwAktRrryEhsooahiRqOJmoY1qGh1yRn3c7ZmjFzghh9XSD5aeWcPZTDuaN5\nnD+aj62TmogYbyIn+ODqK+qDCK4NlhaWuNkocfWOUK+vbyb6JTUlFNUUUVJT0kz0z5Wc43LtZRNb\n39sX+7TDhcsXWLx5MdMCpvFa3GsmGaO/IMsy2oyMZmGYmjNn0FdWAiDZ2KAZMkTx1ocPRzMsCnVw\nEJKF6Wf+dVo9l04Xknwo9//bu7fYRq77juPfQ3I4vFOktKKuuxLX6wt2nTqOncRNmjhuEuShhl2k\nQAIUrdHWcIoaaR6TokhSGIiLGEFfihjo1n0w4KJAm4tdB2lqJ83WTlHHl97stdeXStqudZdIcXi/\nzJw+DCUud6X1elfaIaX/BxjMkBxxjma5P/41c2YO//faOo6jGT4S54aPjnL97RlCMTncIg6e/XHF\n5iXYjs19P7mP2cIsT937FEPhoV3fxn6nHYfG3Fw72NtV++uvo2vuJdq+aJTQ8eOETpxwq/abb8aY\nmNjTro4Vq8HbLy1z5oVF1s6V8PkVUzcPceMdIxw+MYjfvz+7kwlxoX0f4k+8/gTffunbPPzxh7n7\n6N27/v4HlW61qM/MUHv1NWqn3aq9/sYb6KY78IEvmcQ8dh3mdJZgNouZnSaYzWKMje36WKVr7xY5\n8+9LvPXiEtVik3Dc4NjtGW68Y5RDk/3ZhVSIy7WvQ/xc8Ryf/8fPc1vmNr7769/t2Ytg9gvdaFB/\n5x2qr75G7bXXqM/M0JiZwc6fN6p8MEhwaqoT7NNZgtlpzKkpfNGrO75t2w7nTuc488Iis/+zhtPS\nDI5HmbgxTWY6QWY6QTwdks+B2Ff2bYhrrbn/mfs5vX6aJ+95kpHo5d8YS+yuVj5PY3aWxswM9Zn2\nfHaG5rl3wen00w2MjmJOuxV7MDuNmc0SnM4SGD70voO3Vm7yTvtE6PKchd10txNJBLcCPTOdZPhI\nXK4UFX1tz3qneO17b3+PF5de5Bt3fEMC3GOBVIpAKkXk1lu7nncaDZpnz7rBPjvTrtxnKfzgBziV\nzlV3vmj04so9myV4+DAqGNx2m6GowYlPTnDikxPYtkNuvszSTIHlWYvlOYvZ/14D3EGg0mOxrWAf\nmU6SGonIFaNi3+mrSnypvMS9T93L8cHjPPbZx+TP5z6jtaa1stKu3N1gb8zOUJ+do7W42FnR7yc4\nMXFR5W5mp/EPXLobaa3cZHnOYvm8YK9XWgAEQ36GpzqhnplOEI5v/2UhhNf23eEUrTUP/uxBXl5+\nme/f/X0mE5O70DrRK5xymfrcXCfY24dnGnNz6EZnDEp/Ou0G+/Q0xsQkxvg4xvgYxvg4gaGhi7pE\nakezsVJxA33WYmm2wPp8Ge24n/vEUIjMdJKRbILMVJKhyRj+gPSAEd7bd4dTfjTzI56ff56v3v5V\nCfB9yBeNEj5+nPDx7sGstW3TXFi46Lh78Wf/0rkqtU0Fgxijo+1g70zm+DhHs+Pc8JFjKJ+PZsNm\n9WyRpdkCK7MWC2/lefsl99YF/oCPocmYW6lnE2SmEsQH5aSp6F19UYmvVde458l7mE5O8/jnHvf0\npjSidziVCs2FBZrz81vzxvw8zXl32V5f71pfGQbG2NhW5X7+VI8Ns140WJ4rsjxbYPVskVb7pGk4\nEWRETpqKa2xfVeIP//Jhaq0aD33sIQlwscUXiWBedx3mdddt+7pTrV4U8ptBX/v5Key1ta71lWGQ\nGRtlcnwc/9gE5dQUG8ERcg2DtXcvPGka3Qr01GiUVCZCKGZIxS6uuZ4P8WfmnuHZs8/ylVu/QjaZ\n9bo5oo/4wmHMo0cxjx7d9nWnWqW5uLgV7psVfHN+ntq//hx7dY04EAeOAM1wkvLhWygeugFrZYK3\nlwZ4/RedoiIY9pPKRBnIRLqm5HBYxiEVe6anQ3yjtsG3fvktbkrfxH3H7/O6OWKf8YXDmNksZnb7\n4sCp1WgunB/ymxX9izTf/CHN1TVqoTSVSIZKeNidL41yNprhzUD3PfCjYU0ybTAwEiV1OE16IsFA\nJkIsHZI7N4qr0tMh/shLj2DVLU5+5iSGT26CJK4tXyiEmZ3GzE5v+7pTr9NaXKS5skJrZZXW8jKt\nlRVaq29SXclh5VsUqwHKRopKJENxeZjlsxnsVzp3tvNpm6i/SiJik0j5SWUipCZTDF4/Qmwyc03u\nLCn6W89+Qp579zmennmaL33gS9yQvsHr5ghxEZ9purcamJracR2tNY5l0VpZobnsTuWFRfJLZQr5\nJlbZT6kVIl8aYMEaQp/zwcsFoECg+R9Em+vEVJlEuEUi6WdgOMTAeJLQ6DCBTIbA8DD+gYFrcrdJ\n0Zt6NsQbdoMPZT7EAx94wOumCHHFlFL4k0n8ySTmsWMApICJC9bTtk1zdZ3c/y6Sn1snv1CikGtg\nleKsN4aYJwJutsNbDqHaMpHqq0QqK0Tqa8SCDaIxH7EBg1A6SWAwjT892D0fHCSQSu14NazoTz3d\nxVBrLWf7hQAatRaFlSr5eYvc3Dr5eYuNtTqWpWk53VV4wK4RquUwaznMep5QLe/O63nM+gZh08FM\nJfAPpgmkBy+abwX+4CC+REL+D3pkX3QxlA+PEK5gKMChw3EOHY7DHeNbz2utqVgNCitVSvkapXyd\nUq5GMT9Bab1CLlejVr140GCTOiG7iFnYwJxfJWi9Saied8O/nifYsPBpBwIBAul2qKfT2wd/O/D9\n6TS+UOha7hZBj4e4EOLSlFJEkybR5M6DYTcbNuV8nWK+RilX74R9e57P1WjW7O73RRMO2oR9dUJO\niVCjgFleJXhmEWP9DUxrGaNZQl0w1qSKRPAPJPEPDBAYGMB/GZMvHpeC7SpIiAuxzxlB/1af9Z3U\nqy1Kue5wdyt6d76Ur2NHr4coMOb+jM+viEYVEdMh4q8T0mVCTQujtoFRzmFYKwQW3oH8Co5lwU6H\nbv1+97zBjkF/8WuBgQE5tt8mIS6EwAwHMMdjDI7Htn1da02t1KSUr1M8P+zby6v5GuWNENpJuz+g\ngKQ7BY75CCeChCM+wiEwAy1CqoGpqwRbJYK1AqqSw1dchcK6e7HV6dPYGxvoen3HNvsikZ2DP5nA\nl0jiT8TxJ7qXVSSyryr/Kw5xpVQG+InW+oO72B4hRA9SShGOBwnHg+5x+W04jqZabFAtNqhYnal6\n3nKp2GDF0lRLgA4DYeDQ1nsEEj7CE0EiCXcKR3yEDAfT18CkhtkqEWxYGJUcysrjFDZobWxgb2zQ\nmH8Xe6Nw6aofIBDAH4/jS8TxJ5LucjLhLifi+BIJ/PGE+0XQnrvrJPHHYiijt65ZuZpK/Du4/wJC\nCIHP997H5zc5tkO11HQDv9CgUrw49K21KktWg2qpSefQexAYAoYIGG6FH8m2A78d/JFYANNvY6gG\nhl0l2Kpg1ItQsXAsC9sqYlsFnIKFXXSXm4uL2JaFbVnQHk92x98zEnEDPb5Z5Sfa83j3F0EiQeim\nmzBGR692117SFYW4UuouoAws7W5zhBAHgc/v6wT+hZ3mL7AZ+FsBv03gF1arLM0ULgj88xkEgocI\nRccIxQxCSYPQuEEoariPo+4UiQUwAxpD1wm2yvjqJRzrgvC3im7gFy2cgkVzYYHamTdwrCJOqdS1\n1ZE/+yapL35x1/bbdt53iCulgsDXgd8EntxhnQeABwAOHz58Ne0TQhxwXYH/HjYDv7Y5ld2p2l6u\nlzqPi7ma+1yltUPwuydv3aAfIRSddAN/shP6oZg7xaIG4ZiBaSr8dhXKJeyChTG690NIXkkl/jXg\nUa31xk4nB7TWJ4GT4F7sc+XNE0KIy/d+An+T42jqlc3Qb7nBX2pQK7W2vgQ2vxA2litby469Q7Qp\nMCMBwrEgH/4Nm2NDu/TL7eBKQvzTwF1KqQeBW5RSj2mt79/ldgkhxDXh8ynCsSDh2OV3WdRa06zZ\nXSFfPa/yr5eaVMtNQrG9Pwn6vkNca/2JzWWl1CkJcCHEQaOUIhgOEAwHSAx527/jqm59prW+c5fa\nIYQQ4grI/SuFEKKPSYgLIUQfkxAXQog+JiEuhBB9TEJcCCH6mIS4EEL0MQlxIYToY3s+xqZSahU4\nu6cb2XtDwJrXjeghsj86ZF90k/3R7Wr2xxGt9aH3WmnPQ3w/UEq9fDkDlh4Usj86ZF90k/3R7Vrs\nDzmcIoQQfUxCXAgh+piE+OU56XUDeozsjw7ZF91kf3Tb8/0hx8SFEKKPSSUuhBB9TEL8EpRSSaXU\nPymlnlFK/bA9NN2BppTKKKX+0+t29Aql1KNKqbu9bofXlFIppdSPlVIvK6X+yuv2HCQS4pf228Bf\naK0/izso9Oc8bk8v+A7g7V3we4RS6teAEa310163pQf8DvC37e50caXUge1m2C50nm8vG0qpp5VS\n/6aU+v292J6E+CVorR/VWj/bfngIWPGyPV5TSt0FlHG/0A40pZQB/DUwp5S6x+v29IB14IRSagCY\nBM553B5PKKVSwONAtP3Ul4FXtNYfA35LKRXf7W1KiF8GpdQdQEpr/YLXbfFK+1DS13EHyhbwu8Dr\nwCPAh5VSX/a4PV77BXAE+GPgDSDnbXM8YwNfAKz24zuBv28vPwfs+l8oEuLvQSmVBv4S2JM/hfrI\n14BHtdYbXjekR3wQOKm1XgKeAD7lcXu89k3gD7XWDwFngN/zuD2e0FpbWuvCeU9Fgfn2cg7I7PY2\nJcQvoV19/gPwJ1rrfr//y9X6NPCgUuoUcItS6jGP2+O1d4Bse/k2+v/+QFcrBdyslPIDHwGk77Kr\nROccUow9yFwJ8Uv7A+BW4E+VUqeUUl/wukFe0Vp/Qmt9Z3tw7P/SWt/vdZs89jfAp5RSzwF/hHvC\n9yD7c9wLWwpAGvg7b5vTM14BPt5e/hVgbrc3IBf7CCHELlNKndJa36mUOgL8GPgp8KvAR7XW9q5u\nS0JcCCH2jlJqDLca/+cLjpfvzvtLiAshRP+SY+JCCNHHJMSFEKKPSYgLIUQfkxAXQog+JiEuhBB9\n7P8Bm+7Z+QmMsmsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x214843e54e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#产生点集\n",
    "import numpy as np\n",
    "x=np.arange(1,11)\n",
    "\n",
    "y1=[f1(i) for i in x]\n",
    "y2=[f2(i) for i in x]\n",
    "y3=[f3(i) for i in x]\n",
    "y4=[f4(i) for i in x]\n",
    "y5=[f5(i) for i in x]\n",
    "\n",
    "#作图\n",
    "plt.plot(x,y1,label=\"Original VC-bound\")\n",
    "plt.plot(x,y2,label=\"Variant VC-bound\")\n",
    "plt.plot(x,y3,label=\"Rademacher Penalty Bound\")\n",
    "plt.plot(x,y4,label=\"Parrondo and Van den Broek\")\n",
    "plt.plot(x,y5,label=\"Devroye\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到n=5时，Parrando and Van den Broek给出的上界最小。  \n",
    "这题以及上一题可以直接计算，不过我比较喜欢作图比较，感觉更清晰一些。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 6\n",
    "\n",
    "这题和learning from data第二章的Problem 2.3(a)类似\n",
    "\n",
    "第一种情形为$-1,1-1$或者$1,-1,1$，即区间$[l,r]$内部有元素，且区间$[l,r]$两侧均有元素。这种情形只要在$N$个点之间的$N-1$个间隔中挑选出两个即可，所以这种情形共有$2C_{N-1}^2=(N-1)(N-2)$，乘以2是因为$[l,r]$内部可以为-1，也可以为1。\n",
    "\n",
    "第二种情形为$-1,1$或者$1,-1$的，即区间$[l,r]$内部有元素，一侧没有元素。这种情形只要在$N$个点之间的$N-1$个间隔中挑选出一个即可，所以这种情形共有$2(N-1)=2N-2$，乘以2依旧是因为$\\pm 1$的关系。\n",
    "\n",
    "第三种情形为$1$或者$-1$，因此有2种情形。\n",
    "\n",
    "因此\n",
    "$$\n",
    "m_{\\mathcal{H}}(N)=(N-1)(N-2)+2N-2+2=N^2-N+2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 7\n",
    "\n",
    "接着上题，注意\n",
    "$$\n",
    "m_{\\mathcal{H}}(3)=3^2-3+2=8=2^3\n",
    "\\\\m_{\\mathcal{H}}(4)=4^2-4+2=14<2^4\n",
    "$$\n",
    "因此$d_{vc}=3$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 8\n",
    "\n",
    "做映射$y=\\sqrt {x_1^2+ x_2^2}$，所以这个问题可以化为类似Problem 6的问题，但用我之前的思路考虑比较麻烦，这里用另一种思路。注意这里两圆之间的部分为$+1$，因此问题为$N$个点之间以及两侧$N+1$个间隔中挑选2个，最后要需要考虑全$-1$的情形，因此\n",
    "$$\n",
    "m_H (N)=C_{N+1}^2+1\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 9\n",
    "这题就是learning from data第二章的Problem 2.16，结论为$d_{vc}=D+1$\n",
    "\n",
    "这里分两步证明:\n",
    "\n",
    "(a)存在$D+1$个点可以被shatter\n",
    "\n",
    "(b)任意$D+2$个点无法被shatter\n",
    "\n",
    "\n",
    "\n",
    "(a)记$y_j=\\sum_{i=0}^Dc_ix_j^i$，现在取$D+1$个点，$x_1,x_2...x_{D+1},x_j=j$\n",
    "\n",
    "那么\n",
    "$$\n",
    "\\left(\\begin{matrix}\n",
    "y_1\\\\\n",
    "y_2\\\\\n",
    "...\\\\\n",
    "y_{D+1}\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "=\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "\\sum_{i=0}^Dc_ix_1^i\\\\\n",
    "\\sum_{i=0}^Dc_ix_2^i\\\\\n",
    "...\\\\\n",
    "\\sum_{i=0}^Dc_ix_{D+1}^i\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "=\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "1&1&1&...&1\\\\\n",
    "1&2&2^2&...&2^{D}\\\\\n",
    "1&3&3^2&...&3^{D}\\\\\n",
    "...&...&...&...&...&\\\\\n",
    "1&(D+1)&(D+1)^2&...&(D+1)^D\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "\\times\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "c_0\\\\\n",
    "c_1\\\\\n",
    "...\\\\\n",
    "c_D\n",
    "\\end{matrix}\n",
    "\\right)\\\\\n",
    "$$\n",
    "我们知道\n",
    "$$\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "1&1&1&...&1\\\\\n",
    "1&2&2^2&...&2^{D}\\\\\n",
    "1&3&3^2&...&3^{D}\\\\\n",
    "...&...&...&...&...&\\\\\n",
    "1&(D+1)&(D+1)^2&...&(D+1)^D\n",
    "\\end{matrix}\n",
    "\\right)对应的行列式为范德蒙行列式，不为0\n",
    "$$\n",
    "那么对任意的$y=(y_1,y_2,...,y_{D+1})^T$\n",
    "$$\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "1&1&1&...&1\\\\\n",
    "1&2&2^2&...&2^{D}\\\\\n",
    "1&3&3^2&...&3^{D}\\\\\n",
    "...&...&...&...&...&\\\\\n",
    "1&(D+1)&(D+1)^2&...&(D+1)^D\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "\\times\n",
    "\\left(\n",
    "\\begin{matrix}\n",
    "c_0\\\\\n",
    "c_1\\\\\n",
    "...\\\\\n",
    "c_D\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "=\n",
    "\\left(\\begin{matrix}\n",
    "y_1\\\\\n",
    "y_2\\\\\n",
    "...\\\\\n",
    "y_{D+1}\n",
    "\\end{matrix}\n",
    "\\right)\n",
    "$$\n",
    "关于$c=(c_0,c_1,...c_D)^T$有解，因此存在$D+1$个点可以被shatter\n",
    "\n",
    "(b)记$z_j=(1,x_j,x_j^2...x_j^{D})\\in R^{D+1}$，那么现在任取$D+2$个点$x_1,x_2...x_{D+2}$，对应可以得到$z_1,z_2...z_{D+2}$。$R^{D+1}$空间任意$D+2$个点必然线性相关，所以不妨设$z_{D+2}$可以被线性表出，注意$c=(c_0,c_1,...c_D)^T$\n",
    "$$\n",
    "z_{D+2}=\\sum_{i=1}^{D+1}k_iz_i\\\\\n",
    "c^Tz_{D+2}=\\sum_{i=1}^{D+1}c^Tk_iz_i\\\\\n",
    "sign(c^Tz_{D+2})=sign(\\sum_{i=1}^{D+1}k_ic^Tz_i)\n",
    "$$\n",
    "也就是说$z_{D+2}$的分类被$z_1,z_2...z_{D+1}$确定，那么\n",
    "$(sign(z_1),sign(z_2)...sign(z_{D+1}),-sign(z_{D+2}))$这种情形必然无法被划分出来，因此$d_{vc}\\le D+1$\n",
    "\n",
    "结合(a),(b)我们可得\n",
    "$$\n",
    "d_{vc}=D+1\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 10\n",
    "\n",
    "对于$d$维决策树，我们可以理解为将区域划分为$2^d$个互相独立的区域，每个区域可以表示$+1$或者$-1$，所以一共可以表示出$2^{2^d}$种，因此\n",
    "$$\n",
    "d_{vc}=2^d\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 11\n",
    "\n",
    "这题的答案是$+\\infty$，题目挺有意思的，有一题类似的题目可以参考learning from data的Problem 2.18。\n",
    "\n",
    "我们先看题目，乍一看挺复杂的，我们先考虑简单一点的情况，即$\\alpha x$为整数的情形：\n",
    "$$\n",
    "要使得h_{\\alpha}(x)为1，那么|(αx) mod\\ 4 − 2| − 1>0\n",
    "\\\\因此(αx) mod\\ 4=0\n",
    "\\\\要使得h_{\\alpha}(x)为-1，那么|(αx) mod\\ 4 − 2| − 1<0\n",
    "\\\\因此(αx) mod\\ 4=2\n",
    "$$\n",
    "接着考虑一般情形:\n",
    "$$\n",
    "要使得h_{\\alpha}(x)为1，那么|(αx) mod\\ 4 − 2| − 1>0\n",
    "\\\\因此(αx) mod\\ 4>3\n",
    "\\\\或0\\le(αx) mod\\ 4<1\n",
    "\\\\要使得h_{\\alpha}(x)为-1，那么|(αx) mod\\ 4 − 2| − 1<0\n",
    "\\\\因此1<(αx) mod\\ 4<3\n",
    "$$\n",
    "简单来说，当$(\\alpha x)\\ mod 4$处于1到3时$h_{\\alpha}(x)$为$-1$，否则为$+1$。\n",
    "\n",
    "接下来，结合0,2这两个临界点以及上述结论，我们这样构造$\\alpha$以及$x$:\n",
    "\n",
    "任取$(y_1,...,y_N)\\in \\{-1,+1\\}^N$，我们构造这样一个$z=(z_1,...,z_N)$，使得\n",
    "$$\n",
    "当y_i=+1时，z_i=0\n",
    "\\\\ 当y_i=-1时，z_i=2\n",
    "\\\\这一步可以结合我们之前所说的0,2两个特殊情形理解\n",
    "$$\n",
    "现在取\n",
    "$$\n",
    "\\alpha=0.z_1z_2...z_N\\\\\n",
    "x=(x_1 , . . . , x_N), x_n =10^n\n",
    "$$\n",
    "那么\n",
    "$$\n",
    "x_1\\times \\alpha=z_1 .\\ z_{2}...z_{N}\n",
    "\\\\x_i\\times \\alpha=z_1z_2...z_{i-1}z_i\\ .\\ z_{i+1}...z_{N}(i\\ge2)\n",
    "$$\n",
    "首先有一个很显然的结论\n",
    "$$\n",
    "z_1z_2...z_{i-1}z_i\\ mod\\ 4= z_{i-1}z_i\\ mod\\ 4\n",
    "$$\n",
    "这是因为100的倍数模4为0，但这里因为$z_i\\in \\{0,2\\}$，事实上有以下更强的结论\n",
    "$$\n",
    "z_1z_2...z_{i-1}z_i\\ mod\\ 4= z_{i-1}z_i\\ mod\\ 4=z_i\\ mod\\ 4\n",
    "$$\n",
    "这是因为\n",
    "$$\n",
    "z_{i-1}z_i\\ mod\\ 4=\n",
    " \\begin{cases}\n",
    "z_i\\ mod \\ 4(z_{i-1}=0)\n",
    " \\\\(20+z_i)\\ mod \\ 4=z_i\\ mod \\ 4(z_{i-1}=2)\n",
    " \\end{cases}\n",
    "$$\n",
    "因此\n",
    "$$\n",
    "（x_1\\times \\alpha）\\ mod\\ 4=z_1 .\\ z_{2}...z_{N}\\ mod\\ 4\n",
    "\\\\(x_i\\times \\alpha)\\ mod\\ 4=z_1z_2...z_{i-1}z_i\\ .\\ z_{i+1}...z_{N}\\ mod\\ 4=z_i\\ .\\ z_{i+1}...z_{N}\\ mod\\ 4(i\\ge2)\n",
    "\\\\这两式可以统一为\n",
    "\\\\(x_i\\times \\alpha)\\ mod\\ 4=z_i\\ .\\ z_{i+1}...z_{N}\\ mod\\ 4\n",
    "\\\\进一步，我们有\n",
    "\\\\ z_i\\le (x_i\\times \\alpha)\\ mod\\ 4<z_i+1\n",
    "\\\\当z_i=0时，0\\le (x_i\\times \\alpha)\\ mod\\ 4<1\n",
    "\\\\当z_i=2时，2\\le (x_i\\times \\alpha)\\ mod\\ 4<3\n",
    "$$\n",
    "回忆我们之前的结论以及假设\n",
    "$$\n",
    "当(\\alpha x)\\ mod 4处于1到3时，h_{\\alpha}(x)=-1，否则h_{\\alpha}(x)=1\n",
    "\\\\当y_i=+1时，z_i=0\n",
    "\\\\ 当y_i=-1时，z_i=2\n",
    "$$\n",
    "所以\n",
    "$$\n",
    "h_{\\alpha}(x_i)=1=y_i(z_i=0)\n",
    "\\\\h_{\\alpha}(x_i)=-1=y_i(z_i=2)\n",
    "$$\n",
    "所以任意$x=(x_1 , . . . , x_N)$均可以被shatter，因此\n",
    "$$\n",
    "d_{vc}=+\\infty\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 12\n",
    "\n",
    "分两步考虑，第一步先考虑前$i$个点，这$i$个点最多可以区分出$2^i$种，对于剩下的$N-i$个点，最多可以被区分出$m_{\\mathcal{H}}(N-i)$种。那么\n",
    "$$\n",
    "m_{\\mathcal{H}}(N)\\le2^im_{\\mathcal{H}}(N-i)(i=1,2,...N-1)\n",
    "$$\n",
    "因此\n",
    "$$\n",
    "m_{\\mathcal{H}}(N)\\le {min}_{1\\le i\\le N-1}2^im_{\\mathcal{H}}(N-i)(i=1,2,...N-1)\n",
    "$$\n",
    "因此结论成立"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 13\n",
    "\n",
    "这题乍一看挺复杂的，其实想的简单一点就行。\n",
    "\n",
    "我们知道$m_H (N)$要么等于$2^N$，要么有一个多项式的上界，根据这两点，除了$2^{\\lfloor \\sqrt N \\rfloor}$不可能成为$m_H (N)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 14\n",
    "\n",
    "这一题以及下一题均可以参考learning from data的Problem 2.13 \n",
    "\n",
    "(b)首先有直觉反应：\n",
    "$$\n",
    "如果A\\subseteq B,那么d_{vc}(A)\\le d_{vc}(B)\n",
    "$$\n",
    "所以猜测\n",
    "$$\n",
    "0\\le dvc (\\cap _{k=1}^KH_k)\\le {min}\\{(d_{vc}(H_k))\\}_{k=1}^K\n",
    "$$\n",
    "$0\\le d_{vc} (\\cap _{k=1}^KH_k)$是显然的，另一边使用反证法，记$d_1=d_{vc} (\\cap _{k=1}^KH_k),d_2={min}\\{(d_{vc}(H_k))\\}_{k=1}^K$\n",
    "\n",
    "若$d_1\\ge d_2+1$，那么$\\cap _{k=1}^KH_k$可以shatter $d_2+1$个点，那么至少存在一个$H_i(i=1...k)$，使得$H_i$也可以shatter $d_2+1$个点，这就与$d_2={min}\\{(d_{(vc)}(H_k))\\}_{k=1}^K$相矛盾了，所以\n",
    "$$\n",
    "dvc (\\cap _{k=1}^KH_k)\\le {min}\\{(d_{vc}(H_k))\\}_{k=1}^K\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 15\n",
    "\n",
    "(c)这题参考了别人的笔记，[笔记地址](http://beader.me/mlnotebook/section2/vc-dimension-three.html)\n",
    "\n",
    "左边比较简单，由之前的结论即可。\n",
    "$$\n",
    "因为H_k \\subseteq \\cup _{k=1}^KH_k\\\\\n",
    "所以d_{vc}(H_k)\\le d_{vc} (\\cup _{k=1}^KH_k)(k=1...K)\\\\\n",
    "{max}\\{d_{vc}(H_k)\\}_{k=1}^K\\le d_{vc} (\\cup _{k=1}^KH_k)\n",
    "$$\n",
    "观察下右边的形式，其实只要证明$K=2$的情形然后使用数学归纳法即可，下面记$d_1=d_{vc}(H_1),d_2=d_{vc}(H_2)$，考虑成长函数$m_H(N)$，首先\n",
    "$$\n",
    "m _ {H_1\\cup H_2}(N) \\le m_{H_1}(N)+ m_{H_2}(N)\n",
    "$$\n",
    "对于$N$个元素，$H_1\\cup H_2$最多可以表示出$m _ {H_1\\cup H_2}(N)$种分类，对于每种分类，或者由$H_1$表示，或者由$H_2$表示，所以$m _ {H_1\\cup H_2}(N)$应该小于$H_1$和$H_2$表示出来的分类之和。而$H_1$最多可以表示出$m_{H_1}(N)$种分类，$H_2$最多可以表示出$m_{H_2}(N)$种分类，因此上述不等式成立。\n",
    "\n",
    "回到原题，使用上述结论以及VC不等式，我们可得\n",
    "$$\n",
    "m _ {H_1\\cup H_2}(N) \\le m_{H_1}(N)+ m_{H_2}(N)\\le\\sum _ {i=0} ^ {d_1} \\binom{N}{i} + \\sum _ {i=0} ^ {d_2} \\binom{N}{i}\n",
    "$$\n",
    "接着我们使用反证法，如果\n",
    "$$\n",
    "d_{vc}(H_1 \\cup H_2)\\ge 2+d_{vc}(H_1)+d_{vc}(H_2)=d_1+d_2+2\n",
    "$$\n",
    "那么\n",
    "$$\n",
    "m _ {H_1\\cup H_2}(d_1+d_2+2)=2^{d_1+d_2+2}\n",
    "$$\n",
    "把$N=d_1+d_2+2$带入$\\sum _ {i=0} ^ {d_1} \\binom{N}{i} + \\sum _ {i=0} ^ {d_2} \\binom{N}{i}$得\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\sum _ {i=0} ^ {d_1} \\binom{N}{i} + \\sum _ {i=0} ^ {d_2} \\binom{N}{i}&=\\sum _ {i=0} ^ {d_1} \\binom{d_1+d_2+2}{i} + \\sum _ {i=0} ^ {d_2} \\binom{d_1+d_2+2}{i}\n",
    "\\\\&=\\sum _ {i=0} ^ {d_1} \\binom{d_1+d_2+2}{i} + \\sum _ {i=0} ^ {d_2} \\binom{d_1+d_2+2}{d_1+d_2+2-i}\n",
    "\\\\&令j=d_1+d_2+2-i，那么j的范围从d_1+2到d_1+d_2+2\n",
    "\\\\&=\\sum _ {i=0} ^ {d_1} \\binom{d_1+d_2+2}{i} + \\sum _ {j=d_1+2} ^ {d_1+d_2+2} \\binom{d_1+d_2+2}{j}\n",
    "\\\\&=2^{d_1+d_2+2}-\\binom{d_1+d_2+2}{d_1+1}\n",
    "\\\\&<2^{d_1+d_2+2}\n",
    "\\end{aligned}\n",
    "$$\n",
    "所以\n",
    "$$\n",
    "m _ {H_1\\cup H_2}(d_1+d_2+2)\\le\\sum _ {i=0} ^ {d_1} \\binom{d_1+d_2+2}{i} + \\sum _ {i=0} ^ {d_2} \\binom{d_1+d_2+2}{i}<2^{d_1+d_2+2}\n",
    "$$\n",
    "与之前所述矛盾。因此\n",
    "$$\n",
    "d_{vc}(H_1 \\cup H_2)\\le 1+d_{vc}(H_1)+d_{vc}(H_2)=d_1+d_2+1\\\\\n",
    "当K=2时，d_{vc} (\\cup _{k=1}^KH_k)\\le K-1+\\sum_{k=1}^K d_{vc} (H_k)成立\n",
    "$$\n",
    "假设$K=n$时不等式成立，$K=n+1$时\n",
    "$$\n",
    "\\begin{aligned}\n",
    "d_{vc} (\\cup _{k=1}^{n+1}H_k)&=d_{vc} ((\\cup _{k=1}^{n}H_k)\\cup H_{n+1})\n",
    "\\\\&\\le1+d_{vc} (\\cup _{k=1}^{n}H_k)+d_{vc}( H_{n+1})\n",
    "\\\\&\\le1+n-1+\\sum_{k=1}^n d_{vc} (H_k)+d_{vc}( H_{n+1})\n",
    "\\\\&=n+\\sum_{k=1}^{n+1} d_{vc} (H_k)\n",
    "\\end{aligned}\n",
    "$$\n",
    "因此$K=n+1$时不等式也成立"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 16\n",
    "\n",
    "这题可以参考第一题，就是计算数据中带有偏差的$E_{out}(h_s;θ)$，我们取和第一题一致的符号，那么\n",
    "$$\n",
    "E_{out}(h_s;θ)=\\lambda\\mu+(1-\\lambda)(1-\\mu)\n",
    "$$\n",
    "此题中有20%的数据为噪声，因此$\\lambda=0.8$，我们再来计算$\\mu$，即错误率。先回顾题目\n",
    "$$\n",
    "h_{s;θ}(x) = s · sign(x − θ)\n",
    "\\\\当s=1时，若x<\\theta,h_{s;θ}(x)=-1;x>\\theta,h_{s;θ}(x)=1\n",
    "\\\\当s=-1时，若x<\\theta,h_{s;θ}(x)=1;x>\\theta,h_{s;θ}(x)=-1\n",
    "$$\n",
    "我们知道原始数据服从$[-1,1]$上的均匀分布，$\\tilde{s }(x)= sign(x)$。\n",
    "$$\n",
    "因此x<0时,\\tilde{s }(x)=-1;x>0,\\tilde{s }(x)=1\n",
    "$$\n",
    "作图我们可知当$s=1$时，$\\mu=\\frac {|\\theta|}2$，那么\n",
    "$$\n",
    "\\begin{aligned}\n",
    "E_{out}(h_s;θ)&=\\lambda\\mu+(1-\\lambda)(1-\\mu)\n",
    "\\\\&=0.8\\times \\frac{|\\theta|}2+0.2\\times\\frac {2-|\\theta|}2\n",
    "\\\\&=0.2+0.3|\\theta|\n",
    "\\end{aligned}\n",
    "$$\n",
    "当$s=-1$时，$\\mu=\\frac {2-|\\theta|}2$，那么\n",
    "$$\n",
    "\\begin{aligned}\n",
    "E_{out}(h_s;θ)&=\\lambda\\mu+(1-\\lambda)(1-\\mu)\n",
    "\\\\&=0.8\\times\\frac {2-|\\theta|}2+0.2\\times \\frac{|\\theta|}2\n",
    "\\\\&=0.8-0.3|\\theta|\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "\n",
    "我们将$s$和上述两种情况统一在一起可得\n",
    "$$\n",
    "E_{out}(h_s;θ)=0.5+0.3s(|\\theta|-1)\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 17\n",
    "这题以及下一题就是编程模拟上面题目中的情况并计算$E_{in},E_{out}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import math\n",
    "\n",
    "#sign(x)\n",
    "def sign(x):\n",
    "    if x>0:\n",
    "        return 1\n",
    "    else:\n",
    "        return -1\n",
    "    \n",
    "#产生20个随机数\n",
    "def random20():\n",
    "    a=[]\n",
    "    for i in range(20):\n",
    "        a.append(random.uniform(-1,1))\n",
    "    return a\n",
    "\n",
    "#产生(x,sign(x))\n",
    "def dat(a):\n",
    "    b=[]\n",
    "    for i in range(len(a)):\n",
    "        b.append(sign(a[i]))\n",
    "    return b\n",
    "\n",
    "#预测函数\n",
    "def h(x,a,b):\n",
    "    return a*sign(x-b)\n",
    "\n",
    "#产生(x,h(x))预测数据,注意有20%的误差\n",
    "def dat1(x,a,b):\n",
    "    y=[]\n",
    "    for i in range(len(x)):\n",
    "        d=random.random()\n",
    "        if d<0.2:\n",
    "            y.append(-h(x[i],a,b))\n",
    "        else:\n",
    "            y.append(h(x[i],a,b))\n",
    "    return y\n",
    "\n",
    "#产生theta，注意这里只要取20个点相邻两点间的点19个点以及两侧的两个点即可，两侧的点我取了-2以及+2\n",
    "def Theta(x):\n",
    "    a=[-2]\n",
    "    for i in range(len(x)-1):\n",
    "        a.append((x[i]+x[i+1])/2.0)\n",
    "    a.append(2)\n",
    "    return a\n",
    "\n",
    "#计算列表a,b中不同元素个数\n",
    "def cal(a,b):\n",
    "    x=0\n",
    "    for i in range(len(a)):\n",
    "        if a[i]!=b[i]:\n",
    "            x+=1\n",
    "    return x\n",
    "\n",
    "Ein=[]\n",
    "Eout=[]\n",
    "for iteration in range(5000):\n",
    "    ein=0\n",
    "    eout=0\n",
    "    #产生数据\n",
    "    data=random20()\n",
    "    #排序，方便后续处理\n",
    "    data.sort()\n",
    "    #产生(x,sign(x))\n",
    "    y=dat(data)\n",
    "    #产生theta\n",
    "    theta=Theta(data)\n",
    "    #记录最少错误次数，最好的theta以及s\n",
    "    error1=20\n",
    "    theta1=0\n",
    "    s1=1\n",
    "    #注意要考虑s=1以及s=-1\n",
    "    for i in [-1,1]:\n",
    "        for theta2 in theta:\n",
    "            time=0\n",
    "            #产生(x,h(x))预测数据\n",
    "            z=dat1(data,i,theta2)\n",
    "            #计算错误次数\n",
    "            error2=cal(y,z)\n",
    "            if error2<error1:\n",
    "                error1=error2\n",
    "                theta1=theta2\n",
    "                s1=i\n",
    "    #带入之前的公式计算\n",
    "    eout+=0.5+0.3*s1*(abs(theta1)-1)\n",
    "    ein+=error1/(20.0)\n",
    "    Eout.append(eout)\n",
    "    Ein.append(ein)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEFCAYAAAACFke6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEHNJREFUeJzt3X+MZWV9x/H3x11Ill21a5lugIgb\nkk1NU9hUh4WtSxybBUVIJGiypv7TULKNsab/NBEriV0jaExjsyWK3YQ2lEQSxNpqFBdN2bAVKsxC\nBYwSoy6GFcLYJbvZhGCK3/4xh+44zO7cOTN37sw+71dyMuc+55x7nu/c3M88c37cm6pCknRme92o\nOyBJGj7DXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9mpfkz5K8lOT5GdObu2X/muSPR91HabHWjroD\n0grx71X1wdmNVXX9KDojLTVH9pLUAMNeOo0kB5JMzHj8t0n+Psm3k/xPkr0j7J40MA/jSNPel+T5\nbv4nVXXFada9AdgJ/BL4WZI9VXV06D2UFsGwl6bNecz+FL5RVY8CdH8g3gAY9lrRPIwjLdxPZ8z7\nSYJaFQx7aeEMeK06hr007X2zrrN//6g7JC2l+Hn2knTmc2QvSQ0w7CWpAYa9JDXAsJekBqyYm6rO\nPffc2rx586i7IUmryqFDh35VVWPzrbdiwn7z5s1MTk6OuhuStKokeWaQ9TyMI0kNMOwlqQGGvSQ1\nwLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDVgxd9BKK9nmm745kv0e/uw1I9mvzjyO7CWpAYa9\nJDXAsJekBhj2ktQAw16SGmDYS1IDBgr7JJuSHOzm9yQ50E0/TvLxJBckeXZG+1i37h1JHk5y8zCL\nkCSd3rzX2SfZCNwJrAeoqk/OWHYv8C/AZcAtVXX7jGXXA2uqanuSf0qypap+stQFSJLmN8jI/hVg\nF3B8ZmOSS4Fnq+oIcDlwY5LHktzarTIB3NPN3w/smP3ESXYnmUwyOTU11bMESdJ85g37qjpeVcfm\nWPRXwG3d/H1Mh/ulwPYklzD9n8CRbvlRYNMcz72vqsaranxsbN7vy5Uk9dTr4xKS/A7we1X1067p\noap6uVv2OLAFOAGs65ZvwJPBkjQyfQP4fcC3Zjzen+S8JOcAVwFPAYc4eehmK3C4byclSYvT94PQ\n3g383YzHe4AHgF8DX6qqp5M8BxxMcj5wNdPH9SVJIzBw2FfVxIz5P5217AHgrbPajieZAK4EPneK\n4/6SpGUw1I84rqoXOXlFjiRpRDxpKkkNMOwlqQGGvSQ1wLCXpAb4HbRaNUb1PbDSmcCRvSQ1wLCX\npAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lq\nwEBhn2RTkoPd/AVJnk1yoJvGuvY7kjyc5OYZ272mTZK0/OYN+yQbgTuB9V3TZcAtVTXRTVNJrgfW\nVNV24KIkW+ZqG1YRkqTTG2Rk/wqwCzjePb4cuDHJY0lu7domgHu6+fuBHado+y1JdieZTDI5NTXV\nqwBJ0vzmDfuqOl5Vx2Y03cd0kF8KbE9yCdOj/iPd8qPAplO0zX7ufVU1XlXjY2NjvYuQJJ1en68l\nfKiqXgZI8jiwBTgBrOuWb2D6j8hcbZKkEegTwPuTnJfkHOAq4CngECcP02wFDp+iTZI0An1G9nuA\nB4BfA1+qqqeTPAccTHI+cDXTx/VrjjZJ0ggMHPZVNdH9fAB466xlx5NMAFcCn3v1GP9cbZKk5ddn\nZD+nqnqRk1ffnLJNkrT8PGkqSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kN\nMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNWCgsE+yKcnBbv7C\nJAeS/EeSfZl2QZJnu/YDSca6de9I8nCSm4dZhCTp9OYN+yQbgTuB9V3TXwAfrqo/Ad4MXAxcBtxS\nVRPdNJXkemBNVW0HLkqyZTglSJLmM8jI/hVgF3AcoKo+UVU/6pb9LvAr4HLgxiSPJbm1WzYB3NPN\n3w/smP3ESXYnmUwyOTU11b8KSdJpzRv2VXW8qo7Nbk+yC/hhVf0SuI/pcL8U2J7kEqb/EzjSrX4U\n2DTHc++rqvGqGh8bG+tfhSTptNb22SjJRcBfAzu7poeq6uVu2ePAFuAEsK5bvgFPBkvSyCw4gLtj\n+HcDN8wY8e9Pcl6Sc4CrgKeAQ5w8dLMVOLz47kqS+ugzsr8JuBC4LQnAJ4E9wAPAr4EvVdXTSZ4D\nDiY5H7ia6eP6kqQRGDjsq2qi+/kx4GNzrPLWWesfTzIBXAl8bq7j/pKk5dHrmP2gqupFTl6RI0ka\nEU+aSlIDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakB\nhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAQOFfZJNSQ5282cl+UaS7yW5YSFt\nkqTRmDfsk2wE7gTWd00fBQ5V1TuADyR5/QLaJEkjMMjI/hVgF3C8ezwB3NPNPwiML6DttyTZnWQy\nyeTU1NTCey9JGsi8YV9Vx6vq2Iym9cCRbv4osGkBbbOfe19VjVfV+NjYWL8KJEnz6nOC9gSwrpvf\n0D3HoG2SpBHoE8CHgB3d/Fbg8ALaJEkjsLbHNncC30pyBfAHwPeZPlwzSJskaQQGHtlX1UT38xng\nSuB7wM6qemXQtqXuvCRpMH1G9lTVLzl5pc2C2iRJy8+TppLUAMNekhpg2EtSAwx7SWqAYS9JDTDs\nJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16S\nGmDYS1IDen0HbZIPA7u6h78DHGL6y8V/1rV9tKqeTLIHeC/wSFV9ZLGdlST102tkX1W3V9VEVU0A\nB4F/BO5+ta0L+rcDO4BtwAtJdi5ZryVJC7KowzhJLgA2AePAtUkeSXJHkrXAO4GvVlUB+4Er5th+\nd5LJJJNTU1OL6Yok6TQWe8z+I8DtwKPAzqraBpzF9KGb9cCRbr2jTP9R+C1Vta+qxqtqfGxsbJFd\nkSSdSu+wT/I64F3AAeCJqnquWzQJbAFOAOu6tg2L2ZckaXEWE8BXAN/vDtPclWRrkjXAdcAPmD5p\nu6NbdytweDEdlST11+tqnM67gQe7+U8BXwYCfL2qvtuN/D+TZC/wnm6SJI1A77Cvqr+ZMf8UcMms\n5b/prsC5BthbVT/v3UtJ0qIsZmQ/r6p6Cbh3mPuQJM1vqGEvSQu1+aZvjmS/hz97zUj2u1y8QkaS\nGuDIfpUa1egHzvwRkHQmcmQvSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kN\nMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWrAgsM+ydokv0hyoJsuTrInyaNJvjBjvde0\nSZJGo8/I/hLg7qqaqKoJ4GxgB7ANeCHJziRvn922VB2WJC1cn68lvBy4Nsm7gCeBp4GvVlUl2Q9c\nDRybo+27s58oyW5gN8CFF17YswRJ0nz6jOwfBXZW1TbgLGAdcKRbdhTYBKyfo+01qmpfVY1X1fjY\n2FiPrkiSBtFnZP9EVb3czU9yMvABNjD9B+TEHG2SpBHpE8J3JdmaZA1wHdOj+B3dsq3AYeDQHG2S\npBHpM7L/FPBlIMDXgU8DB5PsBd7TTc8An5nVJkkakQWHfVU9xfQVOf+vu9rmGmBvVf38VG2SpNHo\nM7J/jap6Cbh3vjZJ0mh44lSSGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg\n2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqwILDPskbk9yX5P4k\nX0tydpJfJDnQTRd36+1J8miSLyx9tyVJC9FnZP8h4PNVdRXwPHATcHdVTXTTk0neDuwAtgEvJNm5\ndF2WJC3UgsO+qr5YVd/pHo4B/wtcm+SRJHckWQu8E/hqVRWwH7hiyXosSVqw3sfsk2wHNgLfAXZW\n1TbgLOC9wHrgSLfqUWDTKZ5jd5LJJJNTU1N9uyJJmkevsE/yJuA24Abgiap6rls0CWwBTgDrurYN\np9pPVe2rqvGqGh8bG+vTFUnSAPqcoD0b+Arw8ap6BrgrydYka4DrgB8Ah5g+Zg+wFTi8NN2VJPXR\nZ2T/58DbgE8kOQD8ELgL+G/g4ar6LvCfwB8l2Ut3AndpuitJ6mPtQjeoqtuB22c175m1zm+6K3Cu\nAfZW1c/7d1GStFgLDvtBVdVLwL3Den5J0uC8g1aSGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCX\npAYY9pLUAMNekhpg2EtSA4b2cQnLafNN3xzZvg9/9pqR7VuSBuXIXpIaYNhLUgMMe0lqgGEvSQ04\nI07QStJinekXejiyl6QGGPaS1ADDXpIaYNhLUgOGHvZJ7kjycJKbh70vSdLchhr2Sa4H1lTVduCi\nJFuGuT9J0txSVcN78uQfgG9X1beSfBBYV1X/PGP5bmB39/D3gad77upc4FeL6uzqY81tsOY2LKbm\nt1TV2HwrDfs6+/XAkW7+KPC2mQurah+wb7E7STJZVeOLfZ7VxJrbYM1tWI6ah33M/gSwrpvfsAz7\nkyTNYdjhewjY0c1vBQ4PeX+SpDkM+zDOvwEHk5wPXA1cPqT9LPpQ0CpkzW2w5jYMveahnqAFSLIR\nuBJ4sKqeH+rOJElzGnrYS5JGzxOmktSAVRH2g9yFO9c6q/nu3T41J1mb5BdJDnTTxcvX48UbsOZN\nSQ7OeHxWkm8k+V6SG5anp0ujZ70XJHl2xms87/XVK8l8NSd5Y5L7ktyf5GtJzh5ku5WsT83DeC+v\n+LAf5C7cudZZzXfv9q0ZuAS4u6omuunJ5e15fwPWvBG4k+n7N171UeBQVb0D+ECS1y9LhxdpEfVe\nBtwy4zWeWp4eL96A78kPAZ+vqquA54H3nOnvZeaomSG8l1d82AMTwD3d/P2cvJRzvnUG2W6lmqBf\nzZcD1yZ5pBtNrKYvp5lg/ppfAXYBx0+x3YPAarkZZ4J+9V4O3JjksSS3DrWHS2+CeWquqi9W1Xe6\nh2PAC4Nst4JN0K/mJX8vr4awn30X7qYB1xlku5Wqb82PAjurahtwFvDeIfdzKc1bc1Udr6pjC91u\nhepb731MB8ilwPYklwyzk0ts4NcqyXZgY1X910K2W4H61rzk7+XVMPIb5C7cudZZzXfv9q35iap6\nuWubBFbNv7v0f71e3e5Yt92Jpe/aUPSt96FXX+MkjzP9Gj+x9N0bioFqTvIm4Dbg/QvZboXqW/OS\nv5dXwy9tkLtw51pnNd+927fmu5JsTbIGuA74wXC7uaT6vl6r9XXu2+/9Sc5Lcg5wFfDUEPo2LPPW\n3J2Q/Qrw8ap6ZtDtVrC+NS/9e7mqVvQEvKEr9PPAj7pf2KfnWeeNc7WNupZlqPkPmR7lPcn0SbyR\n17KUNc9Y98CM+bcAPwT2Mv2v75pR1zLket8F/Lh7nf9y1HUsdc3Ah4EXgQPdtKuB9/JcNS/5e3lV\n3FQ1yF24c62zmu/e7Vvzata3nu7jOHYA++u1x7hXrDPt9RvEIl7jVfu7Wil9XxVhL0lanNVwzF6S\ntEiGvSQ1wLCXpAYY9pLUAMNekhrwfy8kvQTPxS+gAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e8105486a0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.12504999999999972\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif']=['SimHei'] #用来正常显示中文标签\n",
    "plt.rcParams['axes.unicode_minus']=False #用来正常显示负号\n",
    "\n",
    "plt.hist(Ein)\n",
    "plt.title('Ein')\n",
    "plt.show()\n",
    "\n",
    "print(sum(Ein)/len(Ein))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 18\n",
    "直接看刚刚计算的$E_{out}$即可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEFCAYAAAACFke6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAD/tJREFUeJzt3X2MZXV9x/H3h11otrtWluxkLcR1\nQ7OxMcL6MK6sXXQ0QERsJbQNplZj1WxqiP3Df8RA22BEiWlJ0QjtWtoS2pqAxkdEwJSVrUVltlag\nD4amLtqt1LVQKI21Lf32jzm4s8Ps3nPvztw7s7/3KznZc7/33Ht+39ydz/zmPMykqpAkndhOmvQA\nJEnLz7CXpAYY9pLUAMNekhpg2EtSAwx7SWqAYa8mJXlrkh8meWTe8txJj0taLoa9WvaZqnrOvOW7\nw75BkhcluXg5BictJcNeOj4vAgx7rXiGvdRJclKS301yMMk3k7ysq/9JkrfO2666fw8A1wGXdoeB\nfmsS45b6WDvpAUgT9IYkj3TrDwE3MTdT/xngFcCtSZ5/tBdX1dbum8BMVb11mccqHRfDXi37TFW9\n8ekHST4JfKyq/gv4iySPA2fNf0GSjHmM0pLwMI50pFqwvvA3BZ4xxrFIS8awlw67HXh7kp9I8irg\nVOBB4Ang6csy37XgNT8AngeQZNO4BioNy7BXy94w/zp74DHgAeCfgA8Dv1xVPwI+BvxKki8CBxa8\nxx3A40n+FfjS+IYuDSf+PntJOvE5s5ekBhj2ktQAw16SGmDYS1IDVsxNVZs2baqtW7dOehiStKrs\n37//B1U1NWi7FRP2W7duZXZ2dtLDkKRVJcnDfbbzMI4kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lq\ngGEvSQ0w7CWpAYa9JDVgxdxBe7y2Xn7bRPZ74JqLJrJfSRqGM3tJaoBhL0kNMOwlqQGGvSQ1wLCX\npAYY9pLUgF5hn+S0JOcn2bTcA5IkLb2BYZ9kI/B5YAdwd5KpJDcmuTfJlfO261WTJI1fn5n92cC7\nq+pq4A7gNcCaqtoJnJlkW5JL+tSWqwlJ0rENvIO2qr4MkOSVzM3uTwNu6Z6+E9gFvLhn7aH5751k\nN7AbYMuWLcfRhiTpWPoesw9wKfAYUMDB7qlHgc3A+p61I1TVnqqarqrpqamBfxxdkjSiXmFfcy4D\n7gdeAazrntrQvceTPWuSpAnoc4L2PUne0j08FbiGuUMyANuBA8D+njVJ0gT0+a2Xe4BbkrwDeBD4\nNHBPktOBC4FzmDu0s69HTZI0AX1O0D4GnD+/lmSmq32oqh4fpiZJGr+Rfp999w3gllFqkqTx86Sp\nJDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtS\nAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAQPDPsmz\nk9ye5M4kn0pySpLvJNnbLWd1212V5L4kH5332mfUJEnj12dm/ybg2qq6AHgEuBz4eFXNdMsDSV4K\n7AJ2AN9Pct5itWXqQZI0wMCwr6rrq+qu7uEU8L/A65N8PcmNSdYCrwI+WVUF3AGce5TaEZLsTjKb\nZPbQoUNL1JIkaaHex+yT7AQ2AncB51XVDuBk4HXAeuBgt+mjwOaj1I5QVXuqarqqpqempkZuQpJ0\nbGv7bJTkNOAjwC8Cj1TVj7qnZoFtwJPAuq62gblvIovVJEkT0OcE7SnArcB7q+ph4OYk25OsAS4G\nvgnsZ+74PMB24MBRapKkCegzs3878BLgiiRXAHcDNwMBPltVX0pyEvDBJNcBr+2WhxepSZImYGDY\nV9UNwA0Lylct2Ob/uqttLgKuq6pvAyxWkySNX69j9n1U1Q+BTwyqSZLGz5OmktQAw16SGmDYS1ID\nDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCw\nl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9JDXAsJekBgwM+yTPTnJ7kjuTfCrJKUlu\nTHJvkivnbderJkkavz4z+zcB11bVBcAjwBuBNVW1EzgzybYkl/SpLVcTkqRjWztog6q6ft7DKeBX\ngd/rHt8J7AJeDNzSo/bQ/PdOshvYDbBly5aRGpAkDdb7mH2SncBG4LvAwa78KLAZWN+zdoSq2lNV\n01U1PTU1NVIDkqTBeoV9ktOAjwBvA54E1nVPbejeo29NkjQBfU7QngLcCry3qh4G9jN3SAZgO3Bg\niJokaQIGHrMH3g68BLgiyRXAHwNvTnI6cCFwDlDAvh41SdIEDJzZV9UNVbWxqma65SZgBvgq8Oqq\neryqnuhTW64mJEnH1mdm/wxV9RiHr7QZqiZJGj9PmkpSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QG\nGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBh\nL0kNMOwlqQGGvSQ1wLCXpAYY9pLUgF5hn2Rzkn3d+hlJ/jnJ3m6Z6uo3Jrk3yZXzXveMmiRp/AaG\nfZKNwE3A+q70cuDqqprplkNJLgHWVNVO4Mwk2xarLVcTkqRj6zOzfwq4FHiie3wO8I4kf53kA11t\nBrilW78T2HWUmiRpAgaGfVU9UVWPzyvdzlyQvwzYmeRs5mb9B7vnHwU2H6V2hCS7k8wmmT106NDI\nTUiSjm2UE7R/VVX/UVVPAd8AtgFPAuu65zd077tY7QhVtaeqpqtqempqaoShSJL6GCXs70jy00l+\nErgAeBDYz+HDNNuBA0epSZImYO0Ir7kKuBv4b+D3q+pbSb4H7EtyOnAhc8f1a5GaJGkCeod9Vc10\n/94N/OyC555IMgOcD3zo6WP8i9UkSeM3ysx+UVX1GIevvjlqTZI0ft5BK0kNMOwlqQGGvSQ1wLCX\npAYY9pLUAMNekhpg2EtSAwx7SWqAYS9JDTDsJakBhr0kNcCwl6QGLNkvQmvV1stvm8h+D1xz0UT2\nK2l1cmYvSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1oFfY\nJ9mcZF+3fnKSzyX5SpK3DVOTJE3GwLBPshG4CVjfld4F7K+qnwN+KcmzhqhJkiagz8z+KeBS4Inu\n8QxwS7d+DzA9RO0ISXYnmU0ye+jQoeFHL0nqZWDYV9UTVfX4vNJ64GC3/iiweYjawvfeU1XTVTU9\nNTU1WgeSpIFGOUH7JLCuW9/QvUffmiRpAkYJ4P3Arm59O3BgiJokaQJG+UtVNwFfSHIu8ALga8wd\nrulTkyRNQO+ZfVXNdP8+DJwPfAU4r6qe6ltb6sFLkvoZ6W/QVtW/cPhKm6FqkqTx86SpJDXAsJek\nBhj2ktQAw16SGmDYS1IDDHtJaoBhL0kNMOwlqQGGvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWqA\nYS9JDTDsJakBhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAUOHfZK1Sb6TZG+3\nnJXkqiT3JfnovO2eUZMkTcYoM/uzgY9X1UxVzQCnALuAHcD3k5yX5KULa0s1YEnS8NaO8JpzgNcn\neTXwAPAt4JNVVUnuAC4EHl+k9qWFb5RkN7AbYMuWLSO2IEkaZJSZ/X3AeVW1AzgZWAcc7J57FNgM\nrF+k9gxVtaeqpqtqempqaoShSJL6GGVmf39V/ahbn+Vw4ANsYO4byJOL1CRJEzJKCN+cZHuSNcDF\nzM3id3XPbQcOAPsXqUmSJmSUmf37gD8HAnwWeD+wL8l1wGu75WHggwtqWkJbL79tYvs+cM1FE9u3\npNEMHfZV9SBzV+T8WHe1zUXAdVX17aPVJEmTMcrM/hmq6ofAJwbVJEmT4YlTSWqAYS9JDTDsJakB\nhr0kNcCwl6QGGPaS1ADDXpIaYNhLUgMMe0lqgGEvSQ0w7CWpAYa9JDXAsJekBhj2ktQAw16SGmDY\nS1IDDHtJasCS/KUqtWVSf//Wv30rjc6ZvSQ1wLCXpAYY9pLUAMNekhpg2EtSAwx7SWrAsl96meRG\n4AXAbVX1/uXen05ck7rkE7zsU6vfsoZ9kkuANVW1M8kfJdlWVQ8t5z6l5eC9BSe+E30ykapavjdP\nPgx8saq+kOSNwLqq+uN5z+8GdncPnw986zh2twn4wXG8fqU4UfoAe1mJTpQ+wF6e9ryqmhq00XIf\nxlkPHOzWHwVeMv/JqtoD7FmKHSWZrarppXivSTpR+gB7WYlOlD7AXoa13CdonwTWdesbxrA/SdIi\nljt89wO7uvXtwIFl3p8kaRHLfRjn08C+JKcDFwLnLOO+luRw0ApwovQB9rISnSh9gL0MZVlP0AIk\n2QicD9xTVY8s684kSYta9rCXJE2eJ0xXsCSnJTk/yaZJj0XS6raqwj7JjUnuTXLlUZ5/dpLbk9yZ\n5FNJThn3GPvq0ctG4PPADuDuJAOvo52UQb3M225zkm+Ma1yj6PG5rE3ynSR7u+WscY+xjyE+k+uT\n/Py4xjWKHp/JO+d9Hn+T5A/GPca++nzdJ/lCktml7mPVhP38u3GBM5NsW2SzNwHXVtUFwCPAa8c5\nxr569nI28O6quhq4gwX3KKwUPXt52u9w+FLcFWeIz+XjVTXTLQ+Md5SD9f1MkpwLPKeqPjfWAQ6h\nTy9VdcPTnwewD/jYmIfZS8/P5c3An3XX3D8ryZJde79qwh6YAW7p1u/k8CWdP1ZV11fVXd3DKeD7\n4xna0GYY3MuXq+qrSV7J3Oz+3vENbygzDOgFIMlrgP9k7pvwSjXD4F7OAV6f5OvdLG0l/mnPGQb0\nkeRk5kLxQJI3jG9oQ5uhx/8vgCRnAJuranYM4xrFDIN7+TfghUlOBZ4LfHepdr6awn7h3bibj7Zh\nkp3Axqr66jgGNoJevSQJcCnwGPA/4xna0Ab20h1O+03g8jGOaxR9Ppf7gPOqagdwMvC6MY1tGH36\neAvwd8CHgB1J3jWmsQ2r99c9cBlww7KPaHR9evlL4HnAbwB/3223JFZT2Pe6GzfJacBHgLeNaVyj\n6NVLzbkMuB/4hTGNbVh9erkcuL6q/n1soxpNn17ur6rvdeuzwLEOW01Knz5eDOzpLof+U+DVYxrb\nsPp+3Z/EXA97xzOskfTp5beBX6+q9wH/APzaUu18NYX9wLtxuxnkrcB7q+rh8Q1taH16eU+St3QP\nTwVWalD2uUv6POCyJHuBFyX5w/EMbWh9erk5yfYka4CLgW+OaWzD6NPHPwJnduvTwEr9eul7F/65\nwNdqZV9L3qeXjcBZ3f+vlwNL109VrYoF+CnmvrCuZe7Hm+3A+xds807mDnns7ZZLJz3u4+hlI3AX\ncA9wPd09EStt6dPLgu33TnrMx/m5vJC5n7QeAK6e9JiPo49nMTcxuoe580FnTHrcx/P/C/gAcMmk\nx7sEn8sO4G+Z+yngLmDDUu1/Vd1UdSLdjWsvK9OJ0suJ0gfYy5LtezWFvSRpNKvpmL0kaUSGvSQ1\nwLCXpAYY9pLUAMNekhrw/6QDefNmFiH9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1e8110b49e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2573352820985764\n"
     ]
    }
   ],
   "source": [
    "plt.hist(Eout)\n",
    "plt.title('Eout')\n",
    "plt.show()\n",
    "\n",
    "print(sum(Eout)/len(Eout))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 19\n",
    "\n",
    "这题就是上题多维度的情形，我们要对每个维度分别按上一题的方法计算出最佳的$s,\\theta$，然后对每个维度的最佳解进行比较，求出全局最佳的$s,\\theta$以及维度$dim$，然后按照这个$dim$以及$s,\\theta$来预测我们的测试结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#预处理函数，返回数据以及维度，注意转换的时候不要转换为int  \n",
    "def g(file):\n",
    "    y=[]\n",
    "    with open(file) as x: \n",
    "        for i in x.readlines():\n",
    "            j=i.strip().split(' ')\n",
    "            y.append(list(map(float,j)))\n",
    "        n=len(y[0])-1\n",
    "    return y,n\n",
    "\n",
    "#sign(x)\n",
    "def sign(x):\n",
    "    if x>0:\n",
    "        return 1\n",
    "    else:\n",
    "        return -1\n",
    "    \n",
    "#产生(x,sign(x))\n",
    "def dat(a):\n",
    "    b=[]\n",
    "    for i in range(len(a)):\n",
    "        b.append(sign(a[i]))\n",
    "    return b\n",
    "\n",
    "#预测函数\n",
    "def h(x,a,b):\n",
    "    return a*sign(x-b)\n",
    "\n",
    "#产生(x,h(x))预测数据\n",
    "def dat1(x,a,b):\n",
    "    y=[]\n",
    "    for i in range(len(x)):\n",
    "        y.append(h(x[i],a,b))\n",
    "    return y\n",
    "\n",
    "#产生theta\n",
    "def Theta(x):\n",
    "    a=[]\n",
    "    for i in range(len(x)-1):\n",
    "        a.append((x[i]+x[i+1])/2.0)\n",
    "    x=a[0]-1.0\n",
    "    y=a[-1]+1.0\n",
    "    a.insert(0,x)\n",
    "    a.append(y)\n",
    "    return a\n",
    "\n",
    "#计算列表a,b中不同元素个数\n",
    "def cal(a,b):\n",
    "    x=0\n",
    "    for i in range(len(a)):\n",
    "        if a[i]!=b[i]:\n",
    "            x+=1\n",
    "    return x\n",
    "\n",
    "#行转列，方便后续处理\n",
    "def trans(x):\n",
    "    a=len(x)\n",
    "    b=len(x[0])-1\n",
    "    c=[]\n",
    "    for i in range(b):\n",
    "        d=[]\n",
    "        for j in range(a):\n",
    "            d.append(x[j][i])\n",
    "        c.append(d)\n",
    "    return c\n",
    "\n",
    "def func(x):\n",
    "    #数据，维度\n",
    "    a,b=g(x)\n",
    "    #记录实际值\n",
    "    c=[]\n",
    "    for i in range(len(a)):\n",
    "        c.append(a[i][-1])\n",
    "    #转置\n",
    "    d=trans(a)\n",
    "    #记录最低错误数\n",
    "    error3=len(a)\n",
    "    #记录theta\n",
    "    theta3=0\n",
    "    #记录维度\n",
    "    dim=0\n",
    "    #记录s\n",
    "    s3=1\n",
    "    for i in range(b):\n",
    "        #记录每个维度的最低错误以及theta\n",
    "        error1=len(a)\n",
    "        theta1=0\n",
    "        #记录s\n",
    "        s1=1\n",
    "        #第i个维度\n",
    "        e=d[i]\n",
    "        #排序\n",
    "        data=e[:]\n",
    "        data.sort()\n",
    "        #产生theta\n",
    "        theta=Theta(data)\n",
    "        for j in [-1,1]:\n",
    "            for theta2 in theta:\n",
    "                z=dat1(e,j,theta2)\n",
    "                error2=cal(c,z)\n",
    "                if error2<error1:\n",
    "                    error1=error2\n",
    "                    theta1=theta2\n",
    "                    s1=j\n",
    "        #将第i个维度的最优解error1,theta1,s1和全局最优解比较\n",
    "        if error1<error3:\n",
    "            error3=error1\n",
    "            theta3=theta1\n",
    "            dim=i\n",
    "            s3=s1\n",
    "    #注意维度要加1\n",
    "    return error3/len(a),theta3,dim+1,s3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.25 1.6175000000000002 4 -1\n"
     ]
    }
   ],
   "source": [
    "train='hw2_train.dat'\n",
    "\n",
    "error3,theta3,dim,s3=func(train)\n",
    "\n",
    "print(error3,theta3,dim,s3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$E_{in}=0.25$，维度为第四个维度，$h_{s;i;θ}(x) = s·sign(x_i - θ)=-sign(x_i-1.6175)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 20\n",
    "\n",
    "根据之前计算的$s,\\theta,dim$来预测结果并计算误差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.355\n"
     ]
    }
   ],
   "source": [
    "test='hw2_test.dat'\n",
    "\n",
    "#读取数据\n",
    "test,n=g(test)\n",
    "#数据数量\n",
    "m=len(test)\n",
    "#读取目标值\n",
    "result=[]\n",
    "for i in range(m):\n",
    "    result.append(test[i][-1])\n",
    "#行转列\n",
    "test1=trans(test)\n",
    "#计算预测结果，注意这里选择我们之前的维度\n",
    "predict=dat1(test1[dim-1],s3,theta3)\n",
    "#计算误差\n",
    "error=cal(result,predict)\n",
    "print(error*1.0/m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Problem 21\n",
    "\n",
    "考虑$N$个元素的点集，点集中最多有$k-1$个$-1$，那么这样的点集有$\\sum_{i=0}^{k-1}\\binom N i$种。考虑$k$个元素的子集，然么$(-1,-1...-1)$(全为$-1$的情形)必然无法被表出，因为最多有$k-1$个点为$-1$。那么由$B(N,k)$的定义知\n",
    "$$\n",
    "B(N, k) \\ge\\sum_{i=0}^{k-1}\\binom N i\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### 总结\n",
    "这部分编程题做的比较早，其实解的有点啰嗦，理论题部分有些当时不怎么会，这次又做了一遍算是基本理清楚了。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
